[benbargagliotti]

I keep track of my stocks in an excel worksheet, but this involves logging onto the website, getting my numbers, and punching them in manually. In a few weeks, there will be two weeks when I won't be at a computer every day, and I don't want to lose that much data, so I'm trying to figure out a way to have it all done automatically. i want to import all of my data into a mysql database, and have a few webpages that manipulate the data in different ways (I know how to do all that). The part I don't know how to do is how to render the html from the password protected website to grab my daily info, and import that info into the database. Any good tools out there to do all of this? Thanks

Ben Bargagliotti

[Ludovic Hirlimann]

Originally posted by benbargagliotti:
The part I don't know how to do is how to render the html from the password protected website to grab my daily info, and import that info into the database. Any good tools out there to do all of this? Thanks

Ben Bargagliotti

Curl + the good options + crontab.

Man is your friend.

[Mithras]

Note that curl-lib is accessible from PHP, if it's compiled in (as it is in the Marc Liyange packages).

Then just go to http://www.php.net/curl to read up on how to set the options. Depending on the method of authentication your website uses, it may be easy or difficult to log in using curl. (Easy == HTTP authentication, or cookies. Harder == CGI POST with a Javascript interaction)

Good luck!