Webpage saver ...

Dev Shed Forums Sponsor:

January 13th, 2013, 12:23 PM

cubocik

Registered User

Join Date: Jan 2013

Posts: 14

Time spent in forums: 3 h 31 m 30 sec
Reputation Power: 0

Webpage saver ...

Hey im kind new in this , but I have coded this but It doesn't work...

import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.MalformedURLException;
import java.net.URL;

public class webpagesaver{
public static void main(String[] args) throws IOException{
URLConnection conn = url.openConnection();
conn.connect();
InputStream is = conn.getInputStream();
String saveFile = "PaginaCopiada.html"; //name of SaveFile
String location = "ht#t#p://ww#w.#sa#p#o.com"; // website url withou #
URL url = new URL(location);
BufferedWriter out = new BufferedWriter(new FileWriter(saveFile));
BufferedReader in = new BufferedReader(new InputStreamReader(is));
String l = out.readLine();
while(l!=null){
out.write(l);
out.newLine();
}
in.close();
out.close();

}
}
The objective is : Put website link and Save all the page..

May you help me ?

January 13th, 2013, 03:59 PM

NormR

Contributing User

Join Date: Aug 2010

Location: SW Missouri

Posts: 2,955

Time spent in forums: 1 Week 6 Days 2 h 53 m 34 sec
Reputation Power: 345

Quote:

It doesn't work...

Can you explain what "doesn't work" means?
If you get error messages, please copy the full text and paste it here.
If the program executes but the results are unexpected, please show what it does and explain.

Please edit your post and wrap the code in code tags to preserve formatting.

January 13th, 2013, 04:32 PM

cubocik

Registered User

Join Date: Jan 2013

Posts: 14

Time spent in forums: 3 h 31 m 30 sec
Reputation Power: 0

Code:

 
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.MalformedURLException;
import java.net.URL;

public class webpagesaver{
public static void main(String[] args) throws IOException{


String saveFile = "PaginaCopiada.html"; //name of SaveFile
String location = "http://www.sapo.pt"; // website url withou #
URL url = new URL(location);
URLConnection conn = url.openConnection();
InputStream is = conn.getInputStream();
conn.connect();
BufferedWriter out = new BufferedWriter(new FileWriter(saveFile));
BufferedReader in = new BufferedReader(new InputStreamReader(is));
String l = out.readLine();
while(l!=null){
out.write(l);
out.newLine();
}
in.close();
out.close();

}
}

Compiling messages

Code:

URLConnection conn = url.openConnection();
^
  symbol:   class URLConnection
  location: class webpagesaver
webpagesaver.java:19: error: cannot find symbol
InputStream is = conn.getInputStream();
^
  symbol:   class InputStream
  location: class webpagesaver
webpagesaver.java:23: error: cannot find symbol
String l = out.readLine();
              ^
  symbol:   method readLine()
  location: variable out of type BufferedWriter
3 errors

January 13th, 2013, 05:20 PM

NormR

Contributing User

Join Date: Aug 2010

Location: SW Missouri

Posts: 2,955

Time spent in forums: 1 Week 6 Days 2 h 53 m 34 sec
Reputation Power: 345

Check that there are import statements for the classes referenced in the error messages.

The code needs formatting. Logically nested statements should be indented 3-4 spaces and not start in the first column.

January 14th, 2013, 10:27 AM

cubocik

Registered User

Join Date: Jan 2013

Posts: 14

Time spent in forums: 3 h 31 m 30 sec
Reputation Power: 0

But the idea is that ? Im on the right way ?

January 14th, 2013, 10:54 AM

NormR

Contributing User

Join Date: Aug 2010

Location: SW Missouri

Posts: 2,955

Time spent in forums: 1 Week 6 Days 2 h 53 m 34 sec
Reputation Power: 345

Does the code compile and execute now? It needs to have a clean compile before it can be executed and tested.

It is hard to read and understand unformatted code.

January 14th, 2013, 11:41 AM

cubocik

Registered User

Join Date: Jan 2013

Posts: 14

Time spent in forums: 3 h 31 m 30 sec
Reputation Power: 0

Ty, I checked and corrected

Code:

import java.io.*;
import java.net.*;
 
public class webpagesaver{
    public static void main(String[] args) throws IOException{
    
        String saveFile = "PaginaCopiada.html"; //name of SaveFile
        String location = "http://www.iol.pt"; // website url withou #
        URL url = new URL(location);
        URLConnection conn = url.openConnection();
        InputStream is = conn.getInputStream();
        conn.connect();
        BufferedWriter out = new BufferedWriter(new FileWriter(saveFile));
        BufferedReader in = new BufferedReader(new InputStreamReader(is));
        String l = in.readLine();
        while(l!=null){
            out.write(l);
            out.newLine();
        }
        in.close();
        out.close();
    }    
}

But with that page it works well but for example with facebook it save a blank page, and another example with www.iol.pt its doesn't stop running and I stopped and checked that it created a 1,5Gb its normal ?

January 14th, 2013, 12:05 PM

stdunbar

Contributing User

Join Date: May 2004

Location: Superior, CO, USA

Posts: 2,398

Time spent in forums: 1 Month 2 Weeks 18 h 14 m 16 sec
Reputation Power: 1660

Send a message via Google Talk to stdunbar

How do you read the line from the server? Now how do you read the next line? Hint - debugging 101 might have been to look at the output and realize that you're outputting the same line over and over.

__________________
Need Java help? Want to help people who do? Sit down with a cup of Java at the hotjoe forums.

January 14th, 2013, 12:24 PM

cubocik

Registered User

Join Date: Jan 2013

Posts: 14

Time spent in forums: 3 h 31 m 30 sec
Reputation Power: 0

I dont know , I never work with URL api

#10

January 14th, 2013, 01:24 PM

NormR

Contributing User

Join Date: Aug 2010

Location: SW Missouri

Posts: 2,955

Time spent in forums: 1 Week 6 Days 2 h 53 m 34 sec
Reputation Power: 345

Quote:

I never work with URL api

The problem has nothing to do with the source of the input. The code would have the same problem if it was reading from a disk file.

Change this line:

Code:

 InputStream is = conn.getInputStream();

to this:

 InputStream is = new FileInputStream("<PUT FILENAME HERE>");

compile and execute the code to see what happens.
Replace <PUT FILENAME HERE> with the name of a file on your PC

Last edited by NormR : January 14th, 2013 at 01:28 PM.

#11

January 14th, 2013, 02:44 PM

cubocik

Registered User

Join Date: Jan 2013

Posts: 14

Time spent in forums: 3 h 31 m 30 sec
Reputation Power: 0

The file with the link website ?

#12

January 14th, 2013, 02:57 PM

NormR

Contributing User

Join Date: Aug 2010

Location: SW Missouri

Posts: 2,955

Time spent in forums: 1 Week 6 Days 2 h 53 m 34 sec
Reputation Power: 345

The test I suggested has nothing to do with the web or any website.
It was to show you that the problem with the code has nothing to do with the internet.
The problem will be the same if you try to read a file from your PC.

#13

January 14th, 2013, 03:10 PM

cubocik

Registered User

Join Date: Jan 2013

Posts: 14

Time spent in forums: 3 h 31 m 30 sec
Reputation Power: 0

Ty for the answer I did this

Code:

import java.io.*;
import java.net.*;
 
public class webpagesaver{
    public static void main(String[] args) throws IOException{
    
        String saveFile = "PaginaCopiada.html"; //name of SaveFile
        String location = "http://www.iol.pt"; // website url withou #
        URL url = new URL(location);
        URLConnection conn = url.openConnection();
        InputStream is = new FileInputStream("um.txt"); //conn.getInputStream();
        conn.connect();
        BufferedWriter out = new BufferedWriter(new FileWriter(saveFile));
        BufferedReader in = new BufferedReader(new InputStreamReader(is));
        String l = in.readLine();
        while(l!=null){
            out.write(l);
            out.newLine();
        }
        in.close();
        out.close();
    }    
}

But happens the same

#14

January 14th, 2013, 03:22 PM

NormR

Contributing User

Join Date: Aug 2010

Location: SW Missouri

Posts: 2,955

Time spent in forums: 1 Week 6 Days 2 h 53 m 34 sec
Reputation Power: 345

Yes that is what was supposed to happen. That shows you that the problem is in your program and has nothing to do with the internet.

Did you look in the file that was written?
The problem was explained by stdunbar in post#8.

Where does the code read the next record from the file? It needs to do that inside the loop.

#15

January 14th, 2013, 04:15 PM

cubocik

Registered User

Join Date: Jan 2013

Posts: 14

Time spent in forums: 3 h 31 m 30 sec
Reputation Power: 0

Code:

import java.io.*;
import java.net.*;
 
public class webpagesaver{
    public static void main(String[] args) throws IOException{
    
        String saveFile = "PaginaCopiada.html"; //name of SaveFile
        String location = "http://www.iol.pt"; // website url withou #
        URL url = new URL(location);
        URLConnection conn = url.openConnection();
        InputStream is = conn.getInputStream();
        conn.connect();
        BufferedWriter out = new BufferedWriter(new FileWriter(saveFile));
        BufferedReader in = new BufferedReader(new InputStreamReader(is));
        String l = in.readLine();
        while(l!=null){
            out.write(l);
            out.newLine();
            in.next(); //<-- this ?
        }
        in.close();
        out.close();
    }    
}

Webpage saver ...

Developer Shed Advertisers and Affiliates