Page 1 of 2 12 Last
  • Jump to page:
    #1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2012
    Posts
    17
    Rep Power
    0

    Use zlib to compress and decompress


    I intend to compress some strings and then decompress them. But there are problems about decompression. I have used gdb to debug it, and it showed that the compressed bytes have been stored in the buffer to be ready to decompress, but it isn't compressed completely after the function of uncompress(). I don't know why. Is there some things wrong with my codes of compression.

    Code:
     
    / * 1) compress 3 strings  */
    #include<stdio.h>
    #include<stdlib.h>
    #include<string.h>
    #include"zlib.h"
    unsigned long count = 0;
    unsigned long file_len = 0;
    unsigned long cfile_len = 0;
    unsigned char* compression(const char* s)
    {
    unsigned char *buf;
    unsigned long buflen;
    unsigned long slen = strlen(s) +1;
    buflen = compressBound(slen);
    buf = (unsigned char*)malloc(sizeof(unsigned char)*buflen);
    compress(buf, &buflen, (const Bytef *)s, slen);
    count = buflen;
    file_len = file_len + slen;
    cfile_len = cfile_len + count;
    return buf;
    }
    void main()
    {
    FILE *pf;
    char *p1 = "this document is distributed in the hope that it will be useful, but without any warranty;"; //without even the implied warranty of merchantability or fittness for a particular purpose;You should have received a copy of the GNU General Public License along with this document;";
    / * If the 3 strings are stored together, it can be decompressed completely. But if stored them seperately, only the first one can be decompressed
    */
    char *p2 = "without even the implied warranty of merchantability or fittness for a particular purpose;";
    char *p3 = "You should have received a copy of the GNU General Public License along with this document;";
    pf = fopen("filess.txt", "wb");
    unsigned char* temp;
    temp = compression(p1);
    fwrite(temp, count, 1, pf);
    free(temp);
    printf("%lu\n", count);
    temp = compression(p2);
    fwrite(temp, count, 1, pf);
    free(temp);
    printf("%lu\n", count);
    temp = compression(p3);
    fwrite(temp, count, 1, pf);
    free(temp);
    printf("%lu\n", count);
    printf("%lu\n", file_len); //274
    printf("%lu\n", cfile_len); //251
    fclose(pf);
    }
    /* 2) decompress        this part will be transplant into a "pipe" */
    #include <stdio.h>
    #include <stdlib.h>
    #include <string.h>
    #include <unistd.h>
    #include <sys/types.h>
    #include <sys/stat.h>
    #include <fcntl.h>
    #include "zlib.h"
    #define LINE 1024
    void main()
    {
    char *pf = "filess.txt";
    int p = open(pf, O_RDONLY);
    FILE *nf = fopen("dfile.txt", "wr");
    unsigned char buf[LINE];
    unsigned char dec_buf[LINE*10];
    unsigned long buf_len;
    unsigned long dec_len = LINE * 10;
    buf_len = read(p, buf, LINE);
    uncompress(dec_buf, &dec_len, buf, buf_len); // buf_len == 251, but then it doesn't decompress them all
    fwrite(dec_buf, sizeof(unsigned char), dec_len, nf);
    }
    I don't know why it dosen't decompress completely.
  2. #2
  3. Contributed User
    Devshed Specialist (4000 - 4499 posts)

    Join Date
    Jun 2005
    Posts
    4,363
    Rep Power
    1870
    If you write the output of three compress() calls, then you need three matching uncompress() calls at the other end.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper
  4. #3
  5. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2012
    Posts
    17
    Rep Power
    0
    Originally Posted by salem
    If you write the output of three compress() calls, then you need three matching uncompress() calls at the other end.
    But the work I have to do is to compress hundreds of strings separately, and then decompress them wholly in a file.
  6. #4
  7. Contributed User
    Devshed Specialist (4000 - 4499 posts)

    Join Date
    Jun 2005
    Posts
    4,363
    Rep Power
    1870
    Then you should output the length of each compressed fragment, so that you can correctly call uncompress() later on.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper
  8. #5
  9. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2012
    Posts
    17
    Rep Power
    0

    Smile


    Originally Posted by salem
    Then you should output the length of each compressed fragment, so that you can correctly call uncompress() later on.
    If I call the function of compress() several times, like 3 times, and output all compressed contents in a file, how can I decompress this file? Do I need to use a pointer to mark the position of each fragments in the file and decompress the next one from the pointer? Can you give an example, or just write based on the codes I posted above?

    Thanks!
  10. #6
  11. Contributed User
    Devshed Specialist (4000 - 4499 posts)

    Join Date
    Jun 2005
    Posts
    4,363
    Rep Power
    1870
    Like
    Code:
    temp = compression(p2,&buflen);
    fwrite(&buflen,sizeof(buflen),1,fp);
    fwrite(temp, buflen, 1, pf);
    When you read the file, the first thing you read is a buflen, then read however many bytes buflen tells you to read.
    Then call uncompress()

    Rinse and repeat until the end of the file.

    It shouldn't have taken you 4 days to fail to grok by previous reply, which basically said what to do.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper
  12. #7
  13. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2012
    Posts
    17
    Rep Power
    0
    Originally Posted by salem
    Like
    Code:
    temp = compression(p2,&buflen);
    fwrite(&buflen,sizeof(buflen),1,fp);
    fwrite(temp, buflen, 1, pf);
    When you read the file, the first thing you read is a buflen, then read however many bytes buflen tells you to read.
    Then call uncompress()

    Rinse and repeat until the end of the file.

    It shouldn't have taken you 4 days to fail to grok by previous reply, which basically said what to do.
    Thank you for your replying. But is it the only way to compress and decompress? I need to compress lots of strings which are generated under conditions and output the compressed contents in a file, then decompress the file to get the original strings which is a readable file. Right now, I compressed the strings seperately, and used a pipe to decompress the file, but it seems fail to decompress as a whole file.
  14. #8
  15. Contributed User
    Devshed Specialist (4000 - 4499 posts)

    Join Date
    Jun 2005
    Posts
    4,363
    Rep Power
    1870
    The number of strings are irrelevant to the problem.
    That you're using a pipe is also irrelevant to the problem.
    Standard compression programs handle both these issues, so it's nothing to do with compression per-se that makes such things impossible.

    You need to post some code showing your latest effort.
    Just saying "it doesn't work" isn't good enough.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper
  16. #9
  17. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2012
    Posts
    17
    Rep Power
    0
    Originally Posted by salem
    The number of strings are irrelevant to the problem.
    That you're using a pipe is also irrelevant to the problem.
    Standard compression programs handle both these issues, so it's nothing to do with compression per-se that makes such things impossible.

    You need to post some code showing your latest effort.
    Just saying "it doesn't work" isn't good enough.
    The following is the code I have completed. This programme can decompress a whole file, but can't decompress such file that is composed by several parts of compressed contents. When I use gdb to check it, I found the compressed contents have been put into the buffer, but the function of uncompress() didn't decompress them completely, just decompress the first part, or decompress none of them. I don't know why. I guess if there is any flag in the end when one compression progress has been done, so when it is decompressed, the function of uncompress() will stop as long as detect such flag.

    Code:
    #include<stdio.h>
    #include<stdlib.h>
    #include<string.h>
    #include<sys/types.h>
    #include<sys/stat.h>
    #include<unistd.h>
    #include"zlib.h"
    #include"fcntl.h"
    
    #define LINE 8192
    
    int pipefd[2];
    int pf;
    int fd;
    
    FILE* init_read(char* fil)
    {
    	unsigned long buf_len = 0;
    	unsigned long dec_len = LINE*10;
            char *p;
    	unsigned char buf[LINE], decbuf[LINE*10];
    	unsigned char* temp;
    
    	pf = open(fil, O_RDONLY);
    	if (pf < 0)
    	{
     		printf ("File open error!\n");
     		exit (0);
    	}
    
    	pipe(pipefd);
    	FILE *rpipe = fdopen(pipefd[0], "rb");
    
    	buf_len = read(pf, buf, LINE);
    	uncompress(decbuf, &dec_len, (Bytef*)buf, buf_len);
     	write(pipefd[1], decbuf, dec_len);
    	return rpipe;
    }
    
    int buffer_pipe(FILE *rpipe)
    {
    	char *p;
    	char buf[LINE], decbuf[LINE*10];
    	unsigned char *temp;
    	struct stat tstat;
    	unsigned long buf_len, dec_len = LINE * 10;
    	
    
    	fd = fileno(rpipe);
    	fstat(fd, &tstat);
    	while (tstat.st_size < (2 * LINE))
    	{
    		buf_len = read(pf, buf, LINE);
    		if (buf_len == 0)
    			break;
    		uncompress(decbuf, &dec_len, (Bytef*)buf, buf_len);
    		write(pipefd[1], decbuf, dec_len);
    		fstat(fd, &tstat);
    	}
    
    	return tstat.st_size;	
    }
    
    void close_read(FILE *rpipe)
    {
    	close(pipefd[0]);
    	close(pipefd[1]);
    	fclose(rpipe);
    	return;
    }
    
    void main()
    {
    	char *fname = "filess.txt";
    	char c;
    	int size;
    	FILE *data;
    
    	data = init_read(fname);
    	while (!feof(data))
    	{
    		c = fgetc(data);
    		size = buffer_pipe(data);
    			
    		printf("%c", c);
    	}
    	close_read(data);
    	return;
    }
    Thanks!

    Besides, I post the compression part below. I write a function of compression(), which uses the function of compress() from zlib library. Every time, when I need to compress a string, I call this function.
    Code:
    unsigned char* compression(const char* s)  
    {   
    	unsigned char *buf = NULL;
    	unsigned long buflen = 0;
            unsigned long slen = strlen(s)+1; 
    	buflen = compressBound(slen);
           	buf = (unsigned char*)malloc(sizeof(unsigned char)*buflen);
            compress(buf, &buflen, (const Bytef *)s, slen);
    	compression_count = buflen;
            return buf;
    }
    When I compress a string, I call the fuction of compression(), for example
    Code:
    source = (char*)malloc(sizeof(char)*(1+strlen(top_module->children[0]->types.identifier)+8));
    		sprintf(source, ".model %s\n", top_module->children[0]->types.identifier); 
    		compressed_string = compression(source);
    		fwrite(compressed_string, compression_count, 1, out); // out is a file pointer directing to the file which saves these compressed contents
    
    		free(source);
    		free(compressed_string);
    But I have just found that if I used this compression function to compress a short string, and then use the pipe above to read the decompressed string, it won't show on the screen, is there any bugs in my programme? I have checked the buffer, and the compressed string has been decompress correctly, and the variable "c" gets each letter every time, but why it didn't show on the screen? It is interesting!

    Thanks for your help!
  18. #10
  19. Contributed User
    Devshed Specialist (4000 - 4499 posts)

    Join Date
    Jun 2005
    Posts
    4,363
    Rep Power
    1870
    Getting bored of telling you to write out the length of each compressed block, so that the decompress knows how much data is in each chunk.

    Code:
    #include<stdio.h>
    #include<stdlib.h>
    #include<string.h>
    #include <sys/types.h>
    #include <sys/stat.h>
    #include <fcntl.h>
    #include"zlib.h"
    
    unsigned char* compression(const char* s, unsigned long *buflen)
    {
        unsigned char *buf;
        unsigned long slen = strlen(s) +1;
        *buflen = compressBound(slen);
        buf = malloc(sizeof(unsigned char)* *buflen);
        compress(buf, buflen, (const Bytef *)s, slen);
        return buf;
    }
    
    void decompress(const char *c, unsigned long buflen) {
        unsigned char dec_buf[BUFSIZ];
        unsigned long dec_len = BUFSIZ;
        uncompress(dec_buf, &dec_len, c, buflen);
        write(1,dec_buf,dec_len);
    }
    
    void write_test ( int fd ) {
        char *p[] = {
            "this document is distributed in the hope that it will be useful, but without any warranty;",
            "without even the implied warranty of merchantability or fittness for a particular purpose;",
            "You should have received a copy of the GNU General Public License along with this document;"
        };
        unsigned long len;
        char *result;
        int i;
        for ( i = 0 ; i < 3 ; i++ ) {
            result = compression(p[i],&len);
            write(fd,&len,sizeof(len));
            write(fd,result,len);
            free(result);
        }
    }
    
    void read_test ( int fd ) {
        unsigned long len;
        unsigned char buf[BUFSIZ];
        while ( read(fd,&len,sizeof(len)) == sizeof(len) ) {
            read(fd,buf,len);
            decompress(buf,len);
        }
    }
    
    int main ( int argc, char *argv[] ) {
        int fd;
        if ( argc == 1 ) {
            fprintf(stderr,
                    "Usage: \n"
                    "%s -i [file]   - input from optional file (stdin)\n"
                    "%s -o [file]   - output to optional file (stdout)\n",
                    argv[0], argv[0] );
            return 1;
        }
        switch ( argv[1][1] ) {
            case 'i':
                if ( argc > 2 ) {
                    fd = open(argv[2],O_RDONLY);
                } else {
                    fd = 0;
                }
                read_test(fd);
                break;
            case 'o':
                if ( argc > 2 ) {
                    fd = open(argv[2],O_WRONLY|O_CREAT);
                } else {
                    fd = 1;
                }
                write_test(fd);
                break;
        }
        return 0;    
    }
    Results

    Via a file
    Code:
    $ ./a.out -o zz1
    $ odx zz1
    000000 52 00 00 00 00 00 00 00 78 9c 15 8a d1 0d 80 30  >R.......x......0<
    000010 08 05 1d e5 0d e0 06 4e 83 16 03 49 a5 a6 85 34  >.......N...I...4<
    000020 dd 5e fc ba cb bd e7 a2 03 a5 5d f1 b0 39 7e d7  >.^........]..9~.<
    000030 e1 5d cf 70 2e 50 83 0b 43 da cb 29 94 bb 63 6a  >.].p.P..C..)..cj<
    000040 ad 38 19 31 f8 8e ba 23 9f d9 5c 5a 92 6c 61 52  >.8.1...#..\Z.laR<
    000050 ef 64 be 8e ed 03 13 49 21 62 50 00 00 00 00 00  >.d.....I!bP.....<
    000060 00 00 78 9c 15 c8 db 0d 80 30 08 05 50 47 b9 7b  >..x......0..PG.{<
    000070 38 0d 56 9a 92 f4 41 80 da b8 bd f5 f3 9c 25 51  >8.V...A.......%Q<
    000080 c6 0c f0 c3 1d 51 18 d2 b4 0a df 58 64 46 3d 5e  >.....Q.....XdF=^<
    000090 8c 8c c6 96 ca 06 5d 52 e5 2f 43 96 88 ce ee c8  >......]R./C.....<
    0000a0 1b 04 25 0b 49 b3 92 41 a7 e9 70 3e 8f 0f 44 8d  >..%.I..A..p>..D.<
    0000b0 22 6e 59 00 00 00 00 00 00 00 78 9c 0d ca 31 0e  >"nY.......x...1.<
    0000c0 80 20 10 04 40 9f b2 ff f0 01 34 c6 d8 58 58 e2  >. ..@.....4..XX.<
    0000d0 b1 ca 25 c8 19 01 8d bf d7 a9 67 b1 86 12 ad a5  >..%.......g.....<
    0000e0 80 e8 6f e2 a2 50 6f 06 78 88 9d 2f 6c 43 8d 84  >..o..Po.x../lC..<
    0000f0 1b 67 38 66 5e 3e 61 6a 6b 52 c1 a0 c2 5c 08 9f  >.g8f^>ajkR...\..<
    000100 2c ef 78 b4 c6 7f 6a 41 30 69 07 73 ed bb 0f f6  >,.x...jA0i.s....<
    000110 5e 20 b3                                         >^ .<
    000113
    $ ./a.out -i zz1
    this document is distributed in the hope that it will be useful, but without any warranty;without even the implied warranty of merchantability or fittness for a particular purpose;You should have received a copy of the GNU General Public License along with this document;
    Via a pipe
    Code:
    $ ./a.out -o | ./a.out -i
    this document is distributed in the hope that it will be useful, but without any warranty;without even the implied warranty of merchantability or fittness for a particular purpose;You should have received a copy of the GNU General Public License along with this document;

    Comments on this post

    • asdk77 agrees
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper
  20. #11
  21. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2012
    Posts
    17
    Rep Power
    0

    Smile


    Originally Posted by salem
    Getting bored of telling you to write out the length of each compressed block, so that the decompress knows how much data is in each chunk.

    Code:
    #include<stdio.h>
    #include<stdlib.h>
    #include<string.h>
    #include <sys/types.h>
    #include <sys/stat.h>
    #include <fcntl.h>
    #include"zlib.h"
    
    unsigned char* compression(const char* s, unsigned long *buflen)
    {
        unsigned char *buf;
        unsigned long slen = strlen(s) +1;
        *buflen = compressBound(slen);
        buf = malloc(sizeof(unsigned char)* *buflen);
        compress(buf, buflen, (const Bytef *)s, slen);
        return buf;
    }
    
    void decompress(const char *c, unsigned long buflen) {
        unsigned char dec_buf[BUFSIZ];
        unsigned long dec_len = BUFSIZ;
        uncompress(dec_buf, &dec_len, c, buflen);
        write(1,dec_buf,dec_len);
    }
    
    void write_test ( int fd ) {
        char *p[] = {
            "this document is distributed in the hope that it will be useful, but without any warranty;",
            "without even the implied warranty of merchantability or fittness for a particular purpose;",
            "You should have received a copy of the GNU General Public License along with this document;"
        };
        unsigned long len;
        char *result;
        int i;
        for ( i = 0 ; i < 3 ; i++ ) {
            result = compression(p[i],&len);
            write(fd,&len,sizeof(len));
            write(fd,result,len);
            free(result);
        }
    }
    
    void read_test ( int fd ) {
        unsigned long len;
        unsigned char buf[BUFSIZ];
        while ( read(fd,&len,sizeof(len)) == sizeof(len) ) {
            read(fd,buf,len);
            decompress(buf,len);
        }
    }
    
    int main ( int argc, char *argv[] ) {
        int fd;
        if ( argc == 1 ) {
            fprintf(stderr,
                    "Usage: \n"
                    "%s -i [file]   - input from optional file (stdin)\n"
                    "%s -o [file]   - output to optional file (stdout)\n",
                    argv[0], argv[0] );
            return 1;
        }
        switch ( argv[1][1] ) {
            case 'i':
                if ( argc > 2 ) {
                    fd = open(argv[2],O_RDONLY);
                } else {
                    fd = 0;
                }
                read_test(fd);
                break;
            case 'o':
                if ( argc > 2 ) {
                    fd = open(argv[2],O_WRONLY|O_CREAT);
                } else {
                    fd = 1;
                }
                write_test(fd);
                break;
        }
        return 0;    
    }
    Results

    Via a file
    Code:
    $ ./a.out -o zz1
    $ odx zz1
    000000 52 00 00 00 00 00 00 00 78 9c 15 8a d1 0d 80 30  >R.......x......0<
    000010 08 05 1d e5 0d e0 06 4e 83 16 03 49 a5 a6 85 34  >.......N...I...4<
    000020 dd 5e fc ba cb bd e7 a2 03 a5 5d f1 b0 39 7e d7  >.^........]..9~.<
    000030 e1 5d cf 70 2e 50 83 0b 43 da cb 29 94 bb 63 6a  >.].p.P..C..)..cj<
    000040 ad 38 19 31 f8 8e ba 23 9f d9 5c 5a 92 6c 61 52  >.8.1...#..\Z.laR<
    000050 ef 64 be 8e ed 03 13 49 21 62 50 00 00 00 00 00  >.d.....I!bP.....<
    000060 00 00 78 9c 15 c8 db 0d 80 30 08 05 50 47 b9 7b  >..x......0..PG.{<
    000070 38 0d 56 9a 92 f4 41 80 da b8 bd f5 f3 9c 25 51  >8.V...A.......%Q<
    000080 c6 0c f0 c3 1d 51 18 d2 b4 0a df 58 64 46 3d 5e  >.....Q.....XdF=^<
    000090 8c 8c c6 96 ca 06 5d 52 e5 2f 43 96 88 ce ee c8  >......]R./C.....<
    0000a0 1b 04 25 0b 49 b3 92 41 a7 e9 70 3e 8f 0f 44 8d  >..%.I..A..p>..D.<
    0000b0 22 6e 59 00 00 00 00 00 00 00 78 9c 0d ca 31 0e  >"nY.......x...1.<
    0000c0 80 20 10 04 40 9f b2 ff f0 01 34 c6 d8 58 58 e2  >. ..@.....4..XX.<
    0000d0 b1 ca 25 c8 19 01 8d bf d7 a9 67 b1 86 12 ad a5  >..%.......g.....<
    0000e0 80 e8 6f e2 a2 50 6f 06 78 88 9d 2f 6c 43 8d 84  >..o..Po.x../lC..<
    0000f0 1b 67 38 66 5e 3e 61 6a 6b 52 c1 a0 c2 5c 08 9f  >.g8f^>ajkR...\..<
    000100 2c ef 78 b4 c6 7f 6a 41 30 69 07 73 ed bb 0f f6  >,.x...jA0i.s....<
    000110 5e 20 b3                                         >^ .<
    000113
    $ ./a.out -i zz1
    this document is distributed in the hope that it will be useful, but without any warranty;without even the implied warranty of merchantability or fittness for a particular purpose;You should have received a copy of the GNU General Public License along with this document;
    Via a pipe
    Code:
    $ ./a.out -o | ./a.out -i
    this document is distributed in the hope that it will be useful, but without any warranty;without even the implied warranty of merchantability or fittness for a particular purpose;You should have received a copy of the GNU General Public License along with this document;
    Thank you! Your code can work, but my supervisor wants the compressed file is a standard file which doesn't include other irrelevant information, that is I should not record the length of each string. So right now, I will try to use the function of deflate() and inflate(). I am not sure if this method can work. But I really appreciate your help!!!
  22. #12
  23. Contributed User
    Devshed Specialist (4000 - 4499 posts)

    Join Date
    Jun 2005
    Posts
    4,363
    Rep Power
    1870
    > but my supervisor wants the compressed file is a standard file which doesn't include other irrelevant information, that is I should not record the length of each string.
    OK. so ask them how to solve the problem of separating one string from another, when you've squeezed them all together in a single compressed stream.

    While we're at it, are there any other "you can't use..." or "you must use..." parts of the task which you've so far failed to mention?
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper
  24. #13
  25. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2012
    Posts
    17
    Rep Power
    0

    Smile


    Originally Posted by salem
    > but my supervisor wants the compressed file is a standard file which doesn't include other irrelevant information, that is I should not record the length of each string.
    OK. so ask them how to solve the problem of separating one string from another, when you've squeezed them all together in a single compressed stream.

    While we're at it, are there any other "you can't use..." or "you must use..." parts of the task which you've so far failed to mention?
    I need to compress lots of strings which are generated on conditions, and I should save all the compressed contents in one file. Then I need to decompress the file to get all the original strings. The compressed file should only include the compressed strings so that anyone who decompress the compressed file can get the original strings. Right now, I compress the very string(using the function of compress() in zlib) when it is generated, and save it into a file. But it is a tough work(using the function of uncompress() in zlib) to decompress the file to get the original strings. I am not sure if this problem can be solved by using the functions of inflate() and deflate(), or should I choose another method?

    Thanks!
  26. #14
  27. Contributed User
    Devshed Specialist (4000 - 4499 posts)

    Join Date
    Jun 2005
    Posts
    4,363
    Rep Power
    1870
    So what's wrong with just using strcat() into some large buffer to accumulate ALL your strings, then you use compress() to compress the whole lot in one hit.

    You then use decompress() in one hit to recover a large buffer of all the strings.

    Apparently (from reading the manual - you've read it of course, so you would know this), the low level inflate functions can be used to decompress the kind of files generated by your post #1.

    Study how Z_STREAM_END is returned, and updates to the strm structure to tell you how much data was used from the buffer.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper
  28. #15
  29. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2012
    Posts
    17
    Rep Power
    0

    Smile


    Originally Posted by salem
    So what's wrong with just using strcat() into some large buffer to accumulate ALL your strings, then you use compress() to compress the whole lot in one hit.

    You then use decompress() in one hit to recover a large buffer of all the strings.

    Apparently (from reading the manual - you've read it of course, so you would know this), the low level inflate functions can be used to decompress the kind of files generated by your post #1.

    Study how Z_STREAM_END is returned, and updates to the strm structure to tell you how much data was used from the buffer.
    The original file that isn't compressed can be as large as 30Mb, so how can I allocate a big enough buffer to store all the strings? I will try to use inflate() and learn Z_STREAM_END.
    Thanks a lot!
Page 1 of 2 12 Last
  • Jump to page:

IMN logo majestic logo threadwatch logo seochat tools logo