Q

Automating Web site download process

I am a newbie programmer, primarily working with ASP for use in a mortgage company which I own. I am in search of advice for a project that I am working on. I have checked numerous sites including searchWin2000 without success. I am hoping you may be able to give some direction.

I want to automate the downloading of information from a Web site. Currently, I open a browser and type in the following:
http://drawbridge-red.gte-es.com/drawbridge/nabor/drawbridge.dll?GetListings &user=kMCfnNAB&pass=netKworK&vendor_id=Kmc870∏uct_id=DrKMCfn∝erty_ty pe=0&status=5&max_rows=20&TranslateOptions=TRUE&date_update=02/08/2001

This opens the download manager. I then specify the location to download to. At that point, a flat text file is downloaded to my server's hard disk. From there, I process the text file using SQL Server 2000, which in turn links to numerous ASP applications.

What I am trying to accomplish is automating the actual downloading of this text file. Once I have that program written, I would schedule it using NT Scheduler or SQL Server Agent, thus giving me a completely automated function. Unfortunately, to this point, I have been unsuccessful in finding a solution to automating the download. Any direction you give would be enormously appreciated.


If you are a new programmer, I want you on my team! It sounds like you are doing some interesting development and you are making things happen. Instead of using the browser to download the data, you need to make a small application to do it for you. That way you can control exactly how the data is downloaded. I would recommend that you look at using Visual Basic for the task. It is relatively easy to learn and if you can handle ASP, you can handle Visual Basic. In order to make the downloading easy to program, I would also recommend getting a third party library that encapsulates HTTP and FTP for you. There are numerous companies developing them and any VB related site or magazine has plenty of ads you can use to evaluate. After you get your small download program written, you can also use the scheduler to automate it to run before the SQL 2000 processing.


This was first published in February 2001

Dig deeper on Windows Operating System Management

Pro+

Features

Enjoy the benefits of Pro+ membership, learn more and join.

Have a question for an expert?

Please add a title for your question

Get answers from a TechTarget expert on whatever's puzzling you.

You will be able to add details on the next page.

0 comments

Oldest 

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

-ADS BY GOOGLE

SearchServerVirtualization

SearchCloudComputing

SearchExchange

SearchSQLServer

SearchWinIT

SearchEnterpriseDesktop

SearchVirtualDesktop

Close