Uncategorized (3)


Have you heard of our lord and savior Selenoid?

Last few weeks more and more colleagues at work have been asking for how to work with Selenium. Now i’m of the opinion that System tests (that selenium often are) are expensive, fragile, take time and are mostly a waste of that time. This is because people tend to write WAY more system tests than they would need and really overdo it.

Unit and Integration tests by far more important before we’re putting all of our test-coverage in a system test. With that said, System tests absolutely have their place in a real DevOps environment. As long as you don’t see it as a replacement for manual testing, but more of an indication of the health of your deployment. And what better way to get that in place than using the most populat framework for automating the browser: Selenium.

Selenium

Selenium has bindings in most languages out there which is really neat.
The tricky thing with it is that the framework is decoupled from the drivers that interacts with a particular browser. This makes a’lot of sense because it is easier to provide an interface or protocol (selenese) for the browsers to allow browser vendors to bind their own functionality to match it. So Google Chrome, Mozilla Firefox and Opera, IE has their own drivers, some mobile providers like the android and IOS platforms also have driver implementations that can speak selenese for their systems.

This has a consequence for us as developers that we need to mix and match multiple versions of browsers versions with driver versions and operating systems. This makes selenium tests tricky to set up in a sense that isn’t platform dependent, and as a consequence the CI environment will not match the dev-environment making this kind of testing unreliable.

There’s light in the end of the tunnel though as there’s another kind of driver, the Remote Web Driver. What this does is decouple the need to handle drivers to a server. The driver connects to a service or selenium grid that provides a set of capabilities for which browsers and browser-versions it supports. That way the test does not need to consider setting up the infrastructure For running a selenium test, because that’s the concern of the grid.

So, at work i started looking into setting one of these selenium grids up to help out my colleagues. During that research i stumbled into this video from 2017 SeleniumConf in Berlin about a project called Selenoid.

You can watch it by clicking here.

Hello Selenoid

Selenoid is a project that utilizes Docker images to spin up standalone environments for each running remote selenium session. It is written in Go and also comes with a neat, optional, UI. The UI is able to display the currently running sessions, both its server logs and allow the user to interact with the browser over VNC. This is a disruptor that is in the same market as Saucelabs and Browserstack and makes it possible to get the same scalability and flexibility as these expensive platforms but for free and deployed on your own docker-environment.

Suddenly it has become really easy to spin up a server that takes care of all that is cumbersome with Selenium. Meaning matching driver to browser per environment problem that came with the original java-based selenium server. It also uses a fourth of RAM as the Java version, as claimed in the video.

I really believe Selenoid is the future of running Selenium clusters, i’ve tried it out for some time now and i’m really blown away. So if you are thinking about running a Selenium Server then you should definitely check it out

Vagrant and Selenoid

I’m not a big fan of having a bunch of docker images on my workstation as they usually end up cluttering the machine when i stop using them. So i put sub-projects with docker or other Linux-based experiments in Vagrant-dev environments. Vagrant is a wrapper on top of Virtualbox and makes it real easy to have a completely virtualized environment that can be automatically setup, provisioned and torn down every day. So i wrote a Vagrantfile that downloads an ubuntu/trusty image and provisions Docker, starts Selenoid and Selenoid UI.

I also threw in an example on how to run Remote Selenium tests. You can find it and the vagrantfile and how to run it on my GitHub: Selenoid-Vagrant

private WebDriver browser;

@Before
public void setUp() throws Exception {
    DesiredCapabilities abilities = DesiredCapabilities.chrome();
    abilities.setCapability("version", "61.0");
    this.browser = new RemoteWebDriver( new URL("http://localhost:4444/wd/hub"), abilities);
    this.browser.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
    this.browser.manage().window().setPosition(new Point(220, 10));
    this.browser.manage().window().setSize(new Dimension(1000,650));
}

@Test
public void testSimple() throws Exception {
    this.browser.get("http://www.google.com");
    assertEquals("Google", this.browser.getTitle());
}

@After
public void tearDown() throws Exception {
    this.browser.quit();
}



Just headless system things

I have a RHEL 6.4 Jenkins master that had problems starting an in-build selenium-server using maven. The pom looks something like this. Now i’ve been tracking an error for some time that looks something like this.

[11:43:57] I/launcher - Running 1 instances of WebDriver
[11:43:57] I/hosted - Using the selenium server at http://localhost:4444/wd/hub
[11:44:17] E/launcher - null
[11:44:17] E/launcher - WebDriverError: null
at Object.checkLegacyResponse (/full/path/to/protractor/node_modules/selenium-webdriver/lib/error.js:505:15)

VERY descriptive right?

At first i thought this had something to do with not being able to access the port on localhost, turns out it wasn’t that.

At second glance i thought it had something to do with the webdriver being faulty, so i tried providing something random as a webdriver to see if i had the same problem. Then i got this.

[14:02:34] I/launcher - Running 1 instances of WebDriver
[14:02:34] I/hosted - Using the selenium server at http://localhost:4444/wd/hub
[14:02:35] E/launcher - The driver executable does not exist: C:\bullshit
[14:02:35] E/launcher - WebDriverError: The driver executable does not exist: C:\bullshit

Which is far more descriptive to be honest. But then i remembered that i had piped away the logging from the server somewhere.

<processLogFile>${project.build.directory}/selenium-server/server.log</processLogFile> Aha!

And that’s when i found this lovely line in the logs

Creating a new session for Capabilities [{count=1, browserName=chrome, chromeOptions={args=[--headless, --disable-gpu, --window-size=800,600]}, version=, platform=ANY}]
chromedriver: error while loading shared libraries: libX11.so.6: cannot open shared object file: No such file or directory

Now this i can work with. This is something very common with CentOS, RHEL and Ubuntu Servers. Since they have no X window system all libraries that create something of value for displaying stuff on a screen are skipped out. And apparantly chromium-webdriver requires X11. That feels kind of stupid as my capabilities explicitly specifies running in headless mode. The sane thing would be that this kind of system dependencies shouldn’t be required. But apparantly they do.

Oh well, just get on with installing X11 stuff on your system. This is how you fix it on RHEL/CentOS

yum install libX11

But still it didn’t work!

In the end, it turned out i hadn’t even installed google-chrome-stable in the first place. So i did that, and it worked. However, then i got even worse problems, first libpulse wasn’t installed and was apparantly a requirement for a headless environments (?!).

yum install pulseaudio-libs

This eventually lead to core dumps when trying to make a simple GET towards any site and dump the dom.

[jenkins@my-worker01~]$ /opt/google/chrome/google-chrome --headless --disable-gpu --dump-dom --enable-crash-reporter --enable-logging --v=99 --no-sandbox http://dl.google.com
[1109/092534.785845:VERBOSE1:zygote_main_linux.cc(539)] ZygoteMain: initializing 0 fork delegates
[1109/092534.786417:INFO:cpu_info.cc(50)] Available number of cores: 4
shared memfd open() failed: Invalid argument
[1109/092534.794703:VERBOSE1:pulse_util.cc(138)] Failed to connect to the context. Error: Connection refused
[1109/092534.795643:VERBOSE1:webrtc_internals.cc(109)] Could not get the download directory.
[1109/092534.798643:VERBOSE1:breakpad_linux.cc(2007)] Non Browser crash dumping enabled for: renderer
Failed to generate minidump.Illegal instruction (core dumped)
[0100/000000.826404:ERROR:broker_posix.cc(43)] Invalid node channel message
[jenkins@my-worker01 ~]$

I found at least one other person on the Google Developers Blog where they presented Headless Chrome and that was also a Red Hat Enterprise Linux system.
This is where i ran into a complete halt and decided to file a bug-report towards the chromium project. You can view it below if you want.

https://bugs.chromium.org/p/chromium/issues/detail?id=782161

I’m creating a selenium-cluster with a windows-server instead, at this point it seems easier than configuring chrome-headless for RHEL




New Blog

Hello!

Not that i really had an old one, but it was time to make myself one. Especially considering that fiber was finally installed into our house. Free access to the internet highway means ye olde server actually doing something. So why not a blog, about code and life, at 03:30 AM in the morning. What could possibly go wrong? I most likely misconfigured something along the way.

Anyway. The point of this blog is to keep my code, stories, scripts, failed bread-recipes, some kind of curriculum and misconfigured servers in the same place.

Short introduction.

I’m a father, husband, baker, cyclist, snowboarder, geek and last but not neccessarily least a software engineer/developer/architect. I consider titles to be irrelevant, whoever writes the code really gets the job done. I’m going to keep short on most parts private, shielding my family somewhat from the fierce domain of the intertubes. But professionally i can keep talking.

My profession is first and foremost about writing code. I like functional programming, Linux, the JVM, continuous integration, hosting my own servers and stuff that categorizes as hometech.

In that area there’s more than a few Raspberry Pi’s, one in particular powering a (not yet put inside a box) MagicMirror, which i’m very fond of at the moment.

 

I’m going to make a habit of sharing code snippets i’m particularly fond or proud of, so i’ll get started right away.

Here’s a piece of bash that shows the current git branch (if any), you are on directly on your REPL, give it a spin!

 

Show branch name in terminal

https://gist.github.com/O5ten/1d5f5dda2b3f7abd6194

 

Here’s my GitHub btw, seems like GitHub stopped answering when the site bombarded them with requests while configuring.