Technical Testing Levels—A Call for Standards in Hiring

Brendan Quinn points out why there is a need in the industry for transparent qualifications or a set of qualifications that the industry has trust in. While testing theory is great, hirers want specifics, not aspirational achievements when it comes to technical knowledge and ability.

If you have done hiring in the past then like me you have read CV's over the years where the list of technical abilities of the applicant reads like a person who is a genius. They can code in any language better than a senior programmer, manage any database like a skilled DBA, and design and debug websites like a seasoned web developer. This type of CV is not isolatedit is endemic.

There is too much smoke and mirrors around technical requirements when hiring. They lack detail and specific examples. They request far more than is required in the actual job or completely create a misunderstanding of what the job entails. For an industry built on brain power, we are woefully inadequate at explaining what we want in a QA/test employee, and that probably applies to most IT jobs. This critical analysis of technical job requirements is to create a debate on the subject and offer a possible solution.

Here are sample technical requirements for two real QA jobs I found on the Internet:

Job 1

a) Experience and knowledge of client-side technologies, such as AJAX, XMLHttpRequest, XML, JSON, JSONP, CSS, and JavaScript
b) Knowledge of server-side technologies, such as Java, PHP, and Ruby
c) Understanding of how an OS works, command line tools, and shell scripting
d) Experience with SVN, ANT, GIT, or general configuration management methodologies
e) Ability to create and implement cross-browser HTML/CSS/JS

Job 2

a) Software test experience in web- and Java-based technologies
b) Working knowledge of and demonstrable trouble-shooting skills on Linux/Unix environments
c) Working knowledge and experience of relational databases and SQL
d) Experience with web app automation tools, such as Selenium, Soapui, and Jmeter
e) Knowledge of Python and/or Groovy scripting is desired
f) Proficiency in technologies, such as Javascript, CSS, HTML, Jquery

Now, these technical job requirements are really typical in job postings, but what do they actually mean? Take Job 1: A job applicant could visit Wikipedia to read up on those skills listed in a), memorize the explanations for the interview, and the interviewer might think she has a brilliant match.

Further, what does “experience of” these technologies mean? Does it mean the applicant can write code in all of them or has used each technology in detail? Does it mean the applicant can edit XML files or create new ones that can be used as test data? Maybe the applicant just hung around Ajax developers for a while; would that be experience of Ajax?

Let's pretend a company is hiring a test engineer. The company builds a website using HTML4, CSS, and JavaScript, using a MySQL database, Python scripts, and an Apache web server on a Linux operating system. The website has login authentication, collects users’ data, processes financial information, creates reports, and generates different pageviews depending on the users’ data. It also has an online shop that connects to a warehouse system that processes the orders.

Given the situation above, this is the type of technical job requirements I would like to see:

  • The tester will take ownership over a test environment that replicates the production site.
  • The tester can run various SQL statements like Select, Update, Create, and Delete. The tester can also use MySQL features to achieve above results.
  • The tester can navigate in Linux, do grep-type searches, and execute required files.
  • The tester can edit configuration files to create test data.
  • The tester can use browser tools like Chrome developer tools and Firefox Firebug to locate where problems occur.
  • The tester understands web server/client technology and the basics of programming, so he can communicate with developers on technical issues.

In the categories below, this position would be suitable for a tester with Level 3 capability.

User Comments

Sheila Conway's picture
Sheila Conway

Brendan - Very well-stated and useful article. I appreciate a good tester's knack for asking the question "what do they actually mean", just as you have done here. The types of job descriptions and exams you are advocating for here could help avoid some of the hiring challenges I have run into. A worst case example was multiple candidates in a row web searching terms during in a phone interview for a data warehouse testing position (I could hear them typing) and providing exactly the answers provided off of the top web results. Ouch! Any efforts to raise the quality of QA resources who get hired will eventually raise the standards and perceptions of the QA industry as a whole. Nice work!

January 30, 2013 - 6:09pm
Brendan Quinn's picture
Brendan Quinn

Thanks Sheila. Very glad to hear it isn't just me.

January 31, 2013 - 6:31am

About the author

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.