Pages

[Web Automation] How to automate open multiple website?

Today, we have many websites and many things you must to do manually and it take a lot of your time. You have to repeat your work with a few options. Web automation will help to to simulate your work and save your time.


What's web automation?

Web automation is an open source software with can simulate clicking on the button, filling the form or extracting the data from many websites. It can help you collect data from web and store it to excel file, or from your excel file data you can use it to fill the form and auto submit it with a little effort.

It can connect to SQL Server database to query or create the data, however those functions still not complete. Although incomplete function not working you could write new function to support more data source such as: MongoDB, MySQL or Postgresql...

How to use it?

The web automation application can be downloaded from github, you can add more function and rebuild by using the source code or just download the application and use it normally. It's a portable application so you can bring it to anywhere.

This application required .NET framework 4.5 or higher to work, so please install it first before using web automation application.

After you download the application you must extract it to appropriate place you want, the structure of software will as below:

You need to double click to the WebAutomation.exe to open the application.


So how to use this software to automate click, fill the form or extract data? It's very simple if you following those steps below:
  • Navigate to any website by enter your website in the address bar and the click Go button, wait until your website loaded, please not if the website not complete loaded you won't go to next step because the context menu only show when the web browser complete loaded.
  • Right click any place on the browser, you will see a context menu with have many task you can do such as: Go (navigate to specific website), Sleep (wait in some seconds before continue other method), Extract (extract the data from website), Fill (fill data to textbox or dropdown),...
  • I try first option Go by click to Go item in the menu context, a confirm dialog will show to ask you want to go to the current website or by the link you extract from the current position.
    • For example, I click Yes button to go to the "http://www.google.com.vn", it will show another dialog with the current website, I can change it to another if website or just keep the current website. I keep the current website and click OK
    • The developer tool will show with the go('your website') from previous step, click Run then your website reloaded.
    • Try to change this website to another website and click Run, I change this website from "http://www.google.com.vn" to "https://www.nytimes.com" and click Run the website change from google to the New York Times, you can change any website you want by change the url of this function.
    • If I need to open 2 websites: New York Times and BBC, how I can do that? First keep this code then right click on the website, the context menu will display.
    • You select the Sleep item in the context menu, the Sleep the dialog will display you can enter the time need to wait (for example, I need to wait 5 seconds then I input 5 and click OK)
    • A confirm dialog will display ask you, do you want to wait until the browser loaded or if the browser not loaded in that time you will continue another method.
    • I choose no, because I don't want to wait until the browser load completely, I wait 5 seconds then go to another website. You will see sleep(5, false), if you want to wait until the website loaded try difference number such as 30 seconds, the second parameter will be true. So if the website load only load 15 seconds you don't need to wait 30 seconds to run another function. For example, when I try to navigate to New York Times website and extract the data in this website, I need to wait the website complete loaded then I can extract the data in it, if the website don't complete loaded I can extract the data in this website. But I don't know when the website complete loaded, 5 seconds, 15 seconds or 1 minutes, I don't know. So I will make sleep with longer timer such as 60 seconds it will make sure in 60 seconds if the website load earlier than the current wait we can continue extract data instead of waiting until 1 minute to extract data.
    • I continue right click on the browser then click Go and click Yes in the confirm dialog and enter the BBC website in the next dialog then click OK.
    • Click run you will see the action, the application will navigate to the New York Times, wait 5 seconds and then navigate to bbc.com.

Next tutorial will show you how to click on the link, the button, the checkbox or image...

No comments:

Post a Comment