Page 1 of 2 12 Last
  • Jump to page:
    #1
  1. A Change of Season
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2004
    Location
    Next Door
    Posts
    2,653
    Rep Power
    171

    Htmlespecialchars not converting quotes!


    Hi; What am I doing wrong?

    1 - Why isn't htmlspecialchars converting the quotes?
    2 - Why < has been converted to & l t ; before using htmlspecialchars?
    (Had to add spaces to & l t ; so devshed doesnt convert them)



    Stored in the database as
    Code:
    ".  ' <
    Escape function
    PHP Code:
    return "Raw: ".$var."-------Converted: ".htmlspecialchars($varENT_QUOTESconfig_item('charset')); 
    Html print
    html Code:
    Raw: ". &nbsp;' & l t ;-------Converted: ". &nbsp;' & l t ;
    Last edited by English Breakfast Tea; August 22nd, 2013 at 11:10 PM.
  2. #2
  3. Did you steal it?
    Devshed Supreme Being (6500+ posts)

    Join Date
    Mar 2007
    Location
    Washington, USA
    Posts
    13,993
    Rep Power
    9397
    By "HTML print" do you mean the HTML source of the page you're testing this on? Or what you see on the page?
  4. #3
  5. A Change of Season
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2004
    Location
    Next Door
    Posts
    2,653
    Rep Power
    171
    Originally Posted by requinix
    By "HTML print" do you mean the HTML source of the page you're testing this on? Or what you see on the page?
    Source
  6. #4
  7. A Change of Season
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2004
    Location
    Next Door
    Posts
    2,653
    Rep Power
    171
    Anyone?
  8. #5
  9. No Profile Picture
    Contributing User
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Dec 2004
    Posts
    2,971
    Rep Power
    375
    are you doing "view selection source" in firefox? if so for me it doesnt actually translate. if i do "view source" it seems to be working.

    The only question is what charset you are using? I tested in utf-8 & iso88951 and both converted so is there any charset that it wouldnt convert?
  10. #6
  11. --
    Devshed Expert (3500 - 3999 posts)

    Join Date
    Jul 2012
    Posts
    3,957
    Rep Power
    1046
    Hi,

    the second one sounds like some stupid auto-escape in CodeIgniter. Similar to the infamous "magic quotes" in the old PHP version.

    As to the first question: What happens when you replace $var with a hard-coded string with a single quote?

    PHP Code:
    $var "'";
    var_dumphtmlspecialchars($varENT_QUOTESconfig_item('charset')) ); 
    Last edited by Jacques1; August 23rd, 2013 at 03:18 AM.
    The 6 worst sins of security ē How to (properly) access a MySQL database with PHP

    Why canít I use certain words like "drop" as part of my Security Question answers?
    There are certain words used by hackers to try to gain access to systems and manipulate data; therefore, the following words are restricted: "select," "delete," "update," "insert," "drop" and "null".
  12. #7
  13. A Change of Season
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2004
    Location
    Next Door
    Posts
    2,653
    Rep Power
    171
    Originally Posted by Jacques1
    As to the first question: What happens when you replace $var with a hard-coded string with a single quote?

    PHP Code:
    $var "'";
    var_dumphtmlspecialchars($varENT_QUOTESconfig_item('charset')) ); 
    View source (and what is on screen)
    Code:
    	string(6) "'"
    Originally Posted by Jacques1
    the second one sounds like some stupid auto-escape in CodeIgniter. Similar to the infamous "magic quotes" in the old PHP version.
    What's going on Jaques1? Doesn't really make sense.
  14. #8
  15. --
    Devshed Expert (3500 - 3999 posts)

    Join Date
    Jul 2012
    Posts
    3,957
    Rep Power
    1046
    Originally Posted by English Breakfast Tea
    View source (and what is on screen)
    Code:
    	string(6) "'"
    Is it a literal single quote? The length information says it's 6 bytes. I guess the forum software mangled it again.

    OK, go back to the original code and make a hex dump:

    PHP Code:
    var_dumpbin2hex($var) );
    var_dumpconfig_item('charset') );
    $esc htmlspecialchars($varENT_QUOTESconfig_item('charset')); 
    var_dumpbin2hex($esc) ); 


    Originally Posted by English Breakfast Tea
    What's going on Jaques1? Doesn't really make sense.
    You can turn off the global escaping by setting global_xss_filtering to false. But of course this means you have to escape everything by hand. If you relied on the automatic escaping previously, then your code will be vulnerable.

    Looks like the CI people haven't learned anyhting from the history of PHP.
    The 6 worst sins of security ē How to (properly) access a MySQL database with PHP

    Why canít I use certain words like "drop" as part of my Security Question answers?
    There are certain words used by hackers to try to gain access to systems and manipulate data; therefore, the following words are restricted: "select," "delete," "update," "insert," "drop" and "null".
  16. #9
  17. A Change of Season
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2004
    Location
    Next Door
    Posts
    2,653
    Rep Power
    171
    Hi
    Originally Posted by Jacques1
    Is it a literal single quote? The length information says it's 6 bytes. I guess the forum software mangled it again.

    OK, go back to the original code and make a hex dump:

    PHP Code:
    $var "'";
    var_dumpbin2hex($var) );
    var_dumpconfig_item('charset') );
    $esc htmlspecialchars($varENT_QUOTESconfig_item('charset')); 
    var_dumpbin2hex($esc) ); 
    string(2) "27" string(5) "UTF-8" string(12) "26233033393b"
  18. #10
  19. --
    Devshed Expert (3500 - 3999 posts)

    Join Date
    Jul 2012
    Posts
    3,957
    Rep Power
    1046
    Escaping quotes works perfectly. So in post #7, you saw an HTML entity rather than a single quote on the screen, right?

    Since the function works just like it should, the characters in your database string can't be quotes -- at least not UTF-8 encoded ones. So the question is: What are they then?

    Please make a hex dump of your original $var from the database (the one you used in the "Raw ---- Converted" comparison) and of the escaped result.
    The 6 worst sins of security ē How to (properly) access a MySQL database with PHP

    Why canít I use certain words like "drop" as part of my Security Question answers?
    There are certain words used by hackers to try to gain access to systems and manipulate data; therefore, the following words are restricted: "select," "delete," "update," "insert," "drop" and "null".
  20. #11
  21. A Change of Season
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2004
    Location
    Next Door
    Posts
    2,653
    Rep Power
    171
    Originally Posted by Jacques1
    Escaping quotes works perfectly. So in post #7, you saw an HTML entity rather than a single quote on the screen, right?

    Since the function works just like it should, the characters in your database string can't be quotes -- at least not UTF-8 encoded ones. So the question is: What are they then?

    Please make a hex dump of your original $var from the database (the one you used in the "Raw ---- Converted" comparison) and of the escaped result.
    Please see the attachment to this post Jaquest1
    Attached Images
  22. #12
  23. A Change of Season
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2004
    Location
    Next Door
    Posts
    2,653
    Rep Power
    171
    Jaques1; did you see what I attached above? Still not making sense.
  24. #13
  25. --
    Devshed Expert (3500 - 3999 posts)

    Join Date
    Jul 2012
    Posts
    3,957
    Rep Power
    1046
    ?

    You've already tested a hard-coded single quote in #9, and I've already verified that it works perfectly. The escaped string is the HTML entity for a single quote (& # 0 3 9 . So this part is done.

    I'm interested in the string from the database. In your original post, you said that you escaped a string from the database and didn't get the expected result, right? I'm interested in this very string. Please make a hex dump of the database string (the one you talked about in your original post) and the escaped result.
    The 6 worst sins of security ē How to (properly) access a MySQL database with PHP

    Why canít I use certain words like "drop" as part of my Security Question answers?
    There are certain words used by hackers to try to gain access to systems and manipulate data; therefore, the following words are restricted: "select," "delete," "update," "insert," "drop" and "null".
  26. #14
  27. A Change of Season
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2004
    Location
    Next Door
    Posts
    2,653
    Rep Power
    171
    Originally Posted by Jacques1
    ?

    You've already tested a hard-coded single quote in #9, and I've already verified that it works perfectly. The escaped string is the HTML entity for a single quote (& # 0 3 9 . So this part is done.
    It is all making sense!

    What was confusing was this part:
    PHP Code:
    $data['title'] =  $this->input->post('title') ? html_escape($this->input->post('title')) :  html_escape($data['details'][0]->title); 
    I do NOT need to html_escape the values in the input text! The code above was escaping it twice! This works
    PHP Code:
    $data['title'] =  $this->input->post('title') ? $this->input->post('title') : $data['details'][0]->title
    You did mention this on post #8!

    What I need to learn is the encodings! This specific column is latin1_swedish_ci as in my ap is set to UTF-8. The questions are
    1 - Would this be ok to have database set to latin1_swedish_ci and application to UTF-8?
    2 - Is utf8_general_ci the same as UTF-8? utf8_bin?




    Originally Posted by Jacques1
    I'm interested in the string from the database. In your original post, you said that you escaped a string from the database and didn't get the expected result, right? I'm interested in this very string. Please make a hex dump of the database string (the one you talked about in your original post) and the escaped result.
    I attached a screenshot
    Attached Images
  28. #15
  29. --
    Devshed Expert (3500 - 3999 posts)

    Join Date
    Jul 2012
    Posts
    3,957
    Rep Power
    1046
    Originally Posted by English Breakfast Tea
    The code above was escaping it twice!
    Only because you still haven't turned off the "global escaping".

    Global escaping is stupid. The CI people should have learned from the "magic quotes" disaster, but for some strange reason they decided to make the same mistake the PHP developers did many years ago.

    So the only proper solution is to turn this "feature" off and escape the data right before you use it.



    Originally Posted by English Breakfast Tea
    This works
    I doubt that. If the $data variable isn't covered by the "global escaping", you run right into an XSS vulnerability.



    Originally Posted by English Breakfast Tea
    What I need to learn is the encodings! This specific column is latin1_swedish_ci as in my ap is set to UTF-8. The questions are
    1 - Would this be ok to have database set to latin1_swedish_ci and application to UTF-8?
    2 - Is utf8_general_ci the same as UTF-8? utf8_bin?
    You confuse character encodings and collations.

    UTF-8 and ISO/IEC 8859-1 (aka "latin1") are character encodings. They specify how characters are translated into bit patterns and vice versa. They also determine the character set (the pool of available characters).

    Things like "latin1_swedish_ci" and "utf8_bin" are collations. They specify how strings of a certain character encoding are ordered.

    Those are two different things. The collation is only relevant for ordering and comparing strings. For example, the "ci" collations are case insensitive and will ignore character case. Searching for the string "xy" will also find "XY" and "Xy" and "xY". The "bin" (binary) collations do an exact comparison based on the code point of the character. So searching for "xy" will only find "xy" and nothing else. The details are explained in the manual.

    Yes, you can have an ISO/IEC 8859-1 column and an UTF-8 application. The characters from the database automatically get transcoded into whatever you've specified as the character encoding of the result set. However, I don't see why you'd wanna have to different encodings. This will cause problems when you try to insert Unicode characters outside the ISO/IEC 8859-1 range. It's also inefficient, because the strings have to be translated back and forth whenever you communicate with the database.
    The 6 worst sins of security ē How to (properly) access a MySQL database with PHP

    Why canít I use certain words like "drop" as part of my Security Question answers?
    There are certain words used by hackers to try to gain access to systems and manipulate data; therefore, the following words are restricted: "select," "delete," "update," "insert," "drop" and "null".
Page 1 of 2 12 Last
  • Jump to page:

IMN logo majestic logo threadwatch logo seochat tools logo