#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2003
    Posts
    13
    Rep Power
    0

    Question Maximizing Storage Space using Hex


    Hi all. Bit of a newbie question...

    Can someone give me the Readers Digest version of how to store floats/ints as hex...

    Is it simply stroring a number (say 65325) by pairing up numbers as a hex value? i.e. 0x06 0x53 0x25.

    This is all I've been able to find on the subject so far but, I'm curious as to how floating point values are handled in terms of the decimal point!?!?

    Any feedback will help. Thanks much. :D
    Last edited by nmailey; June 9th, 2003 at 10:10 AM.
  2. #2
  3. Contributing User
    Devshed Supreme Being (6500+ posts)

    Join Date
    Jan 2003
    Location
    USA
    Posts
    7,158
    Rep Power
    2222
    I'm not at all sure what you are trying to ask. As best as I can guess, you're asking about the internal representation of decimal numbers.

    It's all binary. Computers don't do hex, they only do binary (barring binary-coded-decimal (BCD) operations). It has become the convention to organize the binary digits (bits) into groups of 8 called bytes -- the previous generation that developed TCP/IP refered to groupings of 8 bits as "octets" and we still use that term in that context. It turns out that it's very easy to convert a binary number to hexadecimal or to octal -- with a little experience you can do it on sight -- but hex has won out for human-readable output because a byte divides out perfectly into two hex digits, as opposed to 2-2/3 octal digits. BTW, I assume that you are not looking for how to convert between binary and decimal.

    The number of bits used to represent a number determine its maximum values. The most values that can be represented by 8 bits is 2^8 ("two to the eighth power") which is 256. So with one byte you can represent the values 0 to 255 or, -128 to 127. Likewise, two bytes yields 2^16 values: 0 to 65,535 or -32,768 to 32,767.

    BTW, one K is 2^10 which is 1024, which is why 64 K is 65,536. I've seen so many lame explanations as to why a 16 bit processor was called "64K" and I've even seen some data sheets for EEs that called it a "65K" processor. 1 k is 10^3, but 1 K is 2^10. Likewise in binary, one MB is 2^20 bytes and one GB is 2^30 bytes.

    A quick word on byte order. For multi-byte numbers, some computers are "big-endian" and others are "little-endian". That means that they either store the most significant byte first and continue on in descending order (big-endian) or the least significant first and continue on in ascending order (little-endian). Intel processors are little-endian.

    As for floating-point, most follow the IEEE Standard 754, which is described at http://research.microsoft.com/~holl.../ieeefloat.html . Basically, the number is represented as a binary exponent and a binary mantissa. Keep in mind that since floating-point numbers are multi-byte, big-endian/little-endian considerations come into play if you want to examine them in a hex dump or examing/manipulate them with binary operations.

    There was a recent discussion on the subject of floating-point here at http://forums.devshed.com/t64311/s17...eb0ce40a0.html that may help answer your question.
  4. #3
  5. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2003
    Posts
    13
    Rep Power
    0
    I see. I've got the jist on how numbers are stored now. Thanks.

    The problem I'm facing is trying to store 100,000+ "records" in a Palm Pilot. Rather than using a 16 or 32 bit int, I'm looking into maximizing storage by storing these values in (what looks now to be) hex values. I've heard a couple of theories of how this can be done, but can't seem to find anything solid that shows me how. My first post shows one of the methods....but it seems that noone cares about this kind of storage anymore?

    Oh.. I checked out that post earlier this morning...but that looks like some kind of bit-shifting thing no??? As you can see, I've only been programming for a short period. :o
  6. #4
  7. No Profile Picture
    .
    Devshed Newbie (0 - 499 posts)

    Join Date
    Dec 2002
    Posts
    296
    Rep Power
    12
    a hex value is just one way you can choose to represent a number that maybe a, for example, 16 or 32 bit int that you've stored. a quantity is a quantity. it doesn't matter how you then choose to read out those quantities. or a better way to say 'read out' would be represtent.

    i was just looking through a c book and it said a common beginner's confusion is getting a number mixed up with its representation. the book says, they ask questions like "how does a number know if it's a hexadecimal number or a decimal?" and to sort this out, the book shows a picture of a bag of marbles with 17 (that's usual human 17 (decimal)) marbles in - imagine an actual picture of 17 marbles. then this next to it there's:
    Code:
    representation	number of marbles
    ________________________________
    hexadecimal		0x11
    decimal			17
    octal			021
    binary			10001
    the marbles don't care how we count them.
    the point is they're *all the same number*, just different representations. the reason the hex one has got 0x on the start is that's the standard way in c to say "this is a hex representation", and the same with the 0 on the front of the octal (which is base 8) representation. the computer stores this decimal 17 number in the way that it wants and suits it, which when it comes down to it is binary.

    the thing that'll effect the size of all your "records" is the size or range or spectrum or width or depth of each number that you use: the number of possible numbers a number can be (:) that sentance does actually make complete sense i think). that's the 16bit or 32bit specification. basically it comes down to how many possibities do each of your numbers have? what range will they all fit in? eg, if they are all within the range of 0 - 255 for example (and that range can be used in a different way if you want like, -128 to 127 or anyother 256 values you choose it to represent) then you can use an 8 bit number. 100,000+ 8 bit records will take up exactly half the space of 100,000+ 16bit records. a 16bit number has 65536 possabilities (which is a lot more than twice what an 8bit number will give you - that sort of thing increases exponentially). so long as the number of distinct values that all your records have is less that 65536 then you can use 16bit sized numbers.
    Last edited by balance; June 9th, 2003 at 12:54 PM.
  8. #5
  9. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2003
    Posts
    13
    Rep Power
    0
    Right... so we now know that each representation is just different ways of saying the same thing. But isn't there a space advantage somewhere, when using hex?

    For example, If I have a value of 84876....
    If I store that in an unsigned long (allowing the ranges: 0 - 4,294,967,295) (4 bytes), would it not take up less space if I stored 84876 as a char array like this: 0x08 0x48 0x76 ?
    Where the long = 4 bytes & the char = 3? bytes

    I swear I heard storing it this way, or storing values in hex (some other way maybe) allows you to store more information on systems with limited space? Or maybe I misinterpreted things like your marble example somehow.
  10. #6
  11. No Profile Picture
    .
    Devshed Newbie (0 - 499 posts)

    Join Date
    Dec 2002
    Posts
    296
    Rep Power
    12
    Right... so we now know that each representation is just different ways of saying the same thing.
    which as i'm sure you can see is the case in both directions. for both reading the numbers back, and giving the computer the numbers to store. and as has been stated several times the computer stores numbers in binary format - you do not have a choice about that - it's in-built rigid no choice. binary. so when you ask a computer to store a number that's representing 17 (decimal) marbles it lays down 00010001 bits just like that (if you're using 8bit numbers that is) and if you ask the computer to store 0x11 (that's the hexadecimal version of decimal 17) it'll lay down in it's memory 00010001 bits (if you're using 8bit sized numbers).

    also if you ask it to store the number 17 (decimal) to represent the marbles and you're using 16 bit numbers it'll lay it down in it's memory like this: 0000000000010001, or if you ask it to store 0x11 (that's hex for 17 and we're using 16bit numbers still) it'll save in it's memory: 0000000000010001

    it's the bit size that dictates the space (and the possible number of numbers per number) that's used to store each number internally, not the representation you use to give or get back the number. and it's the space that's used internally that's the space in question i think, as opposed to the space on a screen it takes to represent those numbers. as you can see binary takes up more screen space but that isn't the issue i don't think.

    But isn't there a space advantage somewhere, when using hex?
    no, because it doesn't make any difference which representation you use.
  12. #7
  13. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2003
    Posts
    13
    Rep Power
    0
    Well holy crap! Then I've been chasing my tail for the past couple of days. Thanks for the excellent feedback all! :D
  14. #8
  15. Contributing User
    Devshed Supreme Being (6500+ posts)

    Join Date
    Jan 2003
    Location
    USA
    Posts
    7,158
    Rep Power
    2222
    Binary is binary and computers can only understand binary. Humans find binary very hard to understand, so we have come up with numerous ways of representing those binary values, including decimal and hex. How we choose to represent those values makes no difference whatsover with how it gets stored internally.

    However, please note that those human readable representations and not binary, but rather are in characters that are expressed by the computer as character codes, commonly ASCII. Now if you are needing to store ASCII representations of the binary data, then you could express larger values with fewer characters in hex than you could in decimal -- if, of course, you leave off the 0x- prefix. But is that what you are trying to do?

    I'm looking in the O'Reilly "Palm Programming" book (first ed). I don't see any reference to a hex format, nor to any format except binary. In the chapter on writing conduits, they do warn about byte order and point out that the Palm uses a "big-endian" Motorola ÁP which is the reverse of the Wintel "little-endian" ÁP. However, the HotSync Manager provides functions to handle that conversion.

    They also point out that word-alignment differences can lead to wasted space inside of structures (keyword struct), so they recommend using the pack pragma:
    #pragma pack(2)
    You should read more on the pack pragma before you use it.
  16. #9
  17. Contributing User
    Devshed Supreme Being (6500+ posts)

    Join Date
    Jan 2003
    Location
    USA
    Posts
    7,158
    Rep Power
    2222
    Let's try a slightly different tack here.

    There are several compression schemes use with Palm software. Perhaps the term "hex" refers to one of them. However, if that is the case, then they are using the term differently than most programmers think of it, hence the confusion.

    Another thought: I work mainly in embedded software. When I "make" a project on my PC for loading into the target system, what I've made is an image of what will be burned into a PROM. There are several formats for this, including an ASCII file that lists the contents of the PROM in hexadecimal. Then the PROM burner reads that ASCII file and converts it into binary for the actual burning. Could you have been told about something similar?

    Anyway, I found a couple sites that you might find of interest:
    GNU PILOT SDK TUTORIAL at http://www.brouhaha.com/~eric/palm/g..._Tutorial.html

    The PalmOS Open Source Portal at http://www.palmopensource.com/
  18. #10
  19. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2003
    Posts
    13
    Rep Power
    0
    Originally posted by dwise1_aol
    Now if you are needing to store ASCII representations of the binary data, then you could express larger values with fewer characters in hex than you could in decimal -- if, of course, you leave off the 0x- prefix. But is that what you are trying to do?
    Ahhh.... if I'm understanding correctly, yes I think that is
    what I'm trying to do. So you're saying that if I store my ascii
    value of 84876 as the hex value without the '0x' that somehow it
    can be stored in a smaller memory block than the Long Decimal?!?

    I'm getting this idea from our main software which stores dollar
    values and qty's in a non-ascii format to which I've been told was
    to reduce the amount of space it took up when they developed it
    for machines with much less under the hood that what I'm more
    familiar with today.

    Strangly they don't talk about it in that O'Reilly book you
    mentioned. (I've got the 2nd Edition) so I wasn't sure if they did
    it in the background or something. *shrug*.

    I just got those links you shot over...I'll try to take a look and see
    if I can make sense out of things. Thanks.

IMN logo majestic logo threadwatch logo seochat tools logo