Mechanize + Python: How To Follow A Link In A Simple Javascript?

short: How to execute/simulate javascript redirection with python Mechanize? location.href='http://www.site2.com/'; I've made a python script with mechanize module that looks for

Solution 1:

Mechanize can't deal with JavaScript, since it can't interpret it, try parsing your site manually and passing this link to, br.follow_link.

Solution 2:

I solved it! in this way:

    cj = cookielib.LWPCookieJar()
    br.set_cookiejar(cj)

    ...

    br.follow_link("www.address1.com")
    refe= br.geturl()
    req = urllib2.Request(url='www.site2.com')
    req.add_header('Referer', refe)
    opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj) )
    f = opener.open(req) 
    htm = f.read()
    print"\n\n", htm

Solution 3:

How about

br.open("http://alpha.com") 

br.follow_link("http://beta.com")

If you use br_follow_link hopefully that sets the HTTP referrer with the previous page. Whereas if you dobr.open that's like opening a new window, it doesn't set the HTTP referrer header.

---

Edit. Ok it looks like .follow_link doesn't take strings but takes a special mechanize.Link object with a property .absolute_url. You can fake that.

>>>classFake:...pass...>>>x = Fake()>>>x.absolute_url="http://stackoverflow.com">>>br.follow_link(x)
<response_seek_wrapper at 0x2937af8 whose wrapped object = <closeable_response at 0x2937f08 whose fp = <socket._fileobject object at 0x02934970>>>
>>>br.title()
'Stack Overflow'

or make a real mechanize.Link which is less hacky but more tedious.

Solution 4:

You could set the HTTP referrer header explicitly before making your request

br.addheaders = [('Referer', 'http://alpha.com')]
br.open("http://beta.com")

More details in the surprisingly difficult to find official docs http://wwwsearch.sourceforge.net/mechanize/doc.html