Everything You Need To Know About Sikuli A GUI automation tool

Selenium Sikuli for Visual flows automation:

Introduction:

Selenium Sikuli, A tool to automate the GUI test scripts using image recognition method. Sikuli allows users to automate visual work flows using a screenshots (specific method called “Visual Image Match”), in this method all the elements of the webpage are snipped and saved in .png / supported file formats.

Sikuli, by using “Visual Image Match” algorithm, it will match the stored image with the image on the web-elements and performs the scripted action.

Practical Uses:

Ø Useful to automate Flash objects
Ø Automating stable GUI components / Objects / Web-elements
Ø Automating window based applications
Ø Used to automate Audio/Video player, Desktop & Flash games on websites

Sikuli Benefits:

Ø Free Open Source Automation Tool
Ø Automating flash enabled web-elements and objects
Ø Can automate web-elements without ID/Name

Download Sikuli Jar files:

Ø sikulixlibswin-1.1.0 – Click here to download
Ø sikulixsetupAPI-1.1.0-forsetup – Click here to download
Ø sikuli-webdriver-1.0.1 – Click here to download


Click here to get the bundle of above 3 Sikuli automation jar files.

Selenium Sikuli Set-Up:

Ø Download Sikuli jars from above mentioned locations.
Ø Create New project in Eclipse IDE (Refer to this post)
Ø Add Sikuli jar into your project.
Ø After adding Sikuli, we can start with Sikuli scripting.
In order to complete the scripting in Sikuli, we will use some of the Sikuli classes like “Screen” & “Pattern”. These methods are found under “org.sikuli.script” package available in “ sikulixsetupAPI-1.1.0-forsetup”jar file.

About Screen& Pattern method in Selenium Sikuli:

Screen –
Ø This is a class, which will focus on the screen.
Ø To use the “Screen” method, we need to create its object similar to below shown statement.
Screen s1 = new Screen();
Pattern -
Ø This is another class, which will focus on images.
Ø To use the “Pattern” method, we need to create its object similar to below shown statement.
Pattern image_logo = new Pattern("C:\\Users\\Gmail_logo.PNG");

Example using Sikuli – Gmail Logging:

Steps performing in the below script:

Ø Launch Firefox browser (using Selenium code)
Ø Navigate to Google home page. ( https://www.google.com) (using Selenium code)
Ø Click on the “Gmail” (Image-1) displayed on the right top most of the page (using Sikuli code)
Ø Enter email in the email (Image-2) text field (using Sikuli code)
Ø Click Next (Image-3) button (using Sikuli code)
Ø Enter wrong password in the password (Image-4) field (using Sikuli code)
Ø Click on the Sign in (Image-5) button (using Sikuli code)

Images snipped and stored before scripting: (Image Repository)

Image-1: Gmail link saved in .png format

Image-2: Email field

Image-3: Next button

Image-4: Password field

Image-5: Sign in button

Selenium Sikuli – Scripting:

Import Statements:

Web-Driver object creation:

Screen & Pattern object creation:

Launching Firefox browser:

Navigating to “Google”:

Start using Sikuli methods:

Closing the browser:

Click <link>here</link> to download the Selenium Sikuli script.

Selenium Sikuli simple project structure: