[kman42]

I am writing a little Cocoa program that will go out and mine a database. The database has a web interface using a single text box for searches. I basically want my app to do the search once a day and compare the results. I know there are Cocoa classes for retreiving URLS. Are there classes that would let me do a search? Is it just a matter of sending the right URL to the website with the search embedded in the URL?

kman

[Millennium]

Most search engines use the GET method to send the search queries, because that way users can bookmark their searches (you can't do this with the POST method, which is used for almost everything else nowadays).

So if you want to retrieve the page, it's just a matter of embedding your search into the URL. You'll have to play around with the site a bit, to learn the format that it expects the queries in, but that's all on that account.

Once you have the results page, you'll have to crack the actual results out of it. For that, you'll have to parse out the page.

[kman42]

Thanks. Since I know it is possible, I'll get to work on it (I didn't want to waste too much effort if it wasn't even really doable without some backend knowledge of the site). I'll probably be asking some more questions as I go.

kman

[theolein]

You could also just write a script that execs wget or curl and open the resulting file in a browser. Or do I misunderstand what you're trying to do?

[kman42]

A perl script would work well, but I want to make an app that looks nice and that my colleagues will want to use. I want to enable the user to create a list of searches and have the application run the searches on a scheduled interval and return the new results. Just as a bonus I'd like to have the dock icon change whenever there are new results.

I just discovered that the site actually details how to send a search string embedded in a URL. It also offers the flexibility to return the result in several formats including html, text, xml, etc. Needless to say, this little discovery makes my life a whole lot easier. Since they provide the template for the URL, I'll just encode the user's search into a URL and send it off.

Now on to parsing the results! This actually seems pretty straight forward. The results provided by the database look like this:

<DateCreated>
<Year>1977</Year>
<Month>02</Month>
<Day>24</Day>
</DateCreated>

I'm just going to search for the tags and then extract the relevant data contained within.

kman

[kman42]

Do I just deal with retrieving the website as if it were a file on the local system? Using NSURL?

kman

[Millennium]

Originally posted by kman42:
<STRONG>Do I just deal with retrieving the website as if it were a file on the local system? Using NSURL?

kman</STRONG>

You should be able to do this. Make sure the URL is formatted right, though.

As for searching the text, you might want to consider using the XML services. They're very similar to SAX and the DOM (they might even be SAX and the DOM, just with different names for stuff) and it should make things easier on you. Using the XML will keep the page size low, and it'll be easier to parse out (since Cocoa seems to lack regexp's for the time being).

[kman42]

Originally posted by Millennium:
<STRONG>

As for searching the text, you might want to consider using the XML services. They're very similar to SAX and the DOM (they might even be SAX and the DOM, just with different names for stuff) and it should make things easier on you. Using the XML will keep the page size low, and it'll be easier to parse out (since Cocoa seems to lack regexp's for the time being).</STRONG>

Thanks for the info. However, this paragraph was a bit cryptic to me. I am rather new to the programming domain (trying to make an entrance from biology), so some of these abbreviations went right over my head. Would you mind offering more of an explanation on SAX and DOM?

kman