Visual Basic Programming
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed ForumsProgramming Languages - MoreVisual Basic Programming

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
Get inside! Sample the range of functionality easily built with JMSL Library for Time Series Data Analysis, Heat Maps, Portfolio Optimization, Monte Carlo Simulation, Stock Price Charting and more. Download Now!
  #1  
Old October 31st, 2003, 05:21 AM
majormajor majormajor is offline
Junior Member
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Oct 2003
Posts: 2 majormajor User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 0
Babelfish screenscraping

Hi, everyone.

I used to use the xmethods.net babelfish SOAP service to perform an occasional translation via. COM+SOAP/.NET in a couple of personal scripts. Recently (10/03), the xmethod service went under (presumably for good). I'd like to alter my scripts to just screen-scrape the text from a POST response now, much like the Perl module WWW::Babelfish does. Unfortunately, the WScript isn't generating the translations (requesting the page, I suppose) correctly. Peeking in on the POST values via my proxy server seems to indicate that everything is posted correctly, but for some reason, I'm not getting the correct pages back. I'd really prefer to use WScript for this, since WWW::Babelfish would require a complete Perl install on all of my machines, whereas everything I need for the VBScript/WScript stuff is already installed. Any help would be most appreciated. Thank you

This is what my VBScript code looks like (getit.vbs):
Code:
dim xml
set xml = CreateObject("MSXML2.XMLHTTP")
xml.Open "POST", "http://babelfish.altavista.com/babelfish/tr", FALSE
xml.setrequestheader "Referer", "http://babelfish.altavista.com/babelfish/tr"
xml.Send "doit=done&tt=urltext&intl=1&urltext=What+time+do+you+want+to+meet%3F&lp=en_es"
wscript.echo xml.ResponseText


This is what my proxy logs:
Code:
POST /babelfish/tr HTTP/1.0
Accept: */*
Accept-Language: en-us
Referer: URL
Pragma: no-cache
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)
Host: babelfish.altavista.com
Content-Length: 77
Cookie: AV_USERKEY=AVS03b89159d1def80a1c215f0000033; AV_AB=AB11=N&AB12=N&AB13=N&AB49=N&AB46=N&AB47=N&AB48=N; QHist=ModemStatus+00000030,12179|; AV_CLE=v0&1&US:.:.:.:en:.:.:.:.:
Connection: keep-alive
Accept-encoding: gzip, deflate
Browser reload detected...
Posting 77 bytes...
doit=done
tt=urltext
intl=1
urltext=What+time+do+you+want+to+meet%3F
lp=en_es


For comparison, this is what my proxy logs for a normal IE fetch:
Code:
POST /babelfish/tr HTTP/1.0
Accept: */*
Referer: URL
Accept-Language: en-us
Content-Type: application/x-www-form-urlencoded
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)
Host: babelfish.altavista.com
Content-Length: 77
Pragma: no-cache
Cookie: AV_USERKEY=AVS03b89159d1def80a1c215f0000033; AV_AB=AB11=N&AB12=N&AB13=N&AB49=N&AB46=N&AB47=N&AB48=N; QHist=ModemStatus+00000030,12179|; AV_CLE=v0&1&US:.:.:.:en:.:.:.:.:
Connection: keep-alive
Accept-encoding: gzip, deflate
Browser reload detected...
Posting 77 bytes...
doit=done
tt=urltext
intl=1
urltext=What+time+do+you+want+to+meet%3F
lp=en_es


And, finally, what I see when I use WWW::Babelfish:
Code:
POST /babelfish/tr?il=en HTTP/1.0
Host: babelfish.altavista.com
User-Agent: libwww-perl/5.51
Content-Length: 85
Content-Type: application/x-www-form-urlencoded
Connection: keep-alive
Accept-encoding: gzip, deflate
Posting 85 bytes...
doit=done
urltext=What+time+do+you+want+to+meet%3F
lp=en_es
Submit=Translate
enc=utf8

Reply With Quote
  #2  
Old November 1st, 2003, 04:15 AM
cleverpig cleverpig is offline
Contributing User
Dev Shed Beginner (1000 - 1499 posts)
 
Join Date: Jul 2003
Posts: 1,152 cleverpig User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 7
Send a message via MSN to cleverpig

Reply With Quote
  #3  
Old November 1st, 2003, 06:18 AM
majormajor majormajor is offline
Junior Member
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Oct 2003
Posts: 2 majormajor User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 0
The answer was right there in front of me the whole time... msxmlhttp wasn't sending the "Content-Type: application/x-www-form-urlencoded" header. Adding setrequestheader "Content-Type", "application/x-www-form-urlencoded" to my code fixed the problem right up. I should've noticed the problem right off the bat... I guess I just wrongly assumed that indicating the POST method in the Open call would arrange for that.

Thanks very much for your research and response, though.

Reply With Quote
Reply

Viewing: Dev Shed ForumsProgramming Languages - MoreVisual Basic Programming > Babelfish screenscraping


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump


Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 6 hosted by Hostway