Crowdsourced Testing
Published on StickyMinds (https://www.stickyminds.com)

Home > Crowdsourced Testing
QAI Tester's picture

Crowdsourced Testing

QAI Tester asked on October 4, 2013 - 7:42am | Replies (5).

Does Crowdsourced testing provide a much higher ROI compared to the traditional means of Software Testing?

Alex Wilson's picture

I think that traditional paradigme of software testing is overrated. Nobody should keep their own QA department for volatile testing process. Crowdsourced testing looks like much more efficent solution. Application development is not steady, so you have no need to hire a certain number of full-time testers in your company. Instead of this you could hire testing professionals just at time they are really needed. one of such services we personally use in our company is Ubertesters 'Hire-testers' service - it's like we set up our own team but on-demand basis. there are lots of similar services you can find and try as well. And of course it provides a much higher ROI compared to the traditional means of testing.. with crowdtesting you don't have 'fixed costs' when you have nothing to test

Simon Papineau's picture

My company recently did a comparative study of crowdsourced vs. outsourced testing.

What we found is that crowdsourced testing yields 4.7x the amount of bugs in 60% of the time.

That being said, not every type of project is a suitable candidate for crowdsourced testing.

As its been said by fellow respondents, a hybrid model of internal and external testing is still the best approach for maximum value and efficiency.

However, crowdsourced testing can definity contribute significant ROI while helping reduced the cost of software testing for development companies.

 

Mukesh Sharma's picture

Not really. It all depends on what kind of testing tasks you have on hand for which you plan to use crowdsourced testing. It is very important to decide what, when and how to crowdsource in a test effort failing which the ROI can be very poor and can shake the entire team’s confidence in the model. When the right choices are made on these areas (for example, using the crowd to test for an application’s usability across devices), the ROI will most often be higher than the traditional testing engagement models. In short though, we are still at a stage where crowdsourced testing is a supplement to the traditional testing models and together the two can yield a very high ROI on the testing effort, when the effort is well planned and implemented. Co-incidentally, my collegue and I did a tutorial at Star West last year on this topic "Implementing a Successful Crowdsourced Testing Effort"

Renuka Bhatnagar's picture

In my opinion, Crowdsourced testing can have a higher ROI if testing requires various platforms and environments with minimal data scenarios that are required to validate all business rules.

 

I have worked on an application where data scenarios are very complex and to validate every functionality, we needed to consider multiple data combinatios. Such an application would probably not give you higher return in crowdsourced environment. We need internal testers who knows the applciation and can either create or match the data conditions to validate all scenarios.

5 Answers

Renuka Bhatnagar's picture

In my opinion, Crowdsourced testing can have a higher ROI if testing requires various platforms and environments with minimal data scenarios that are required to validate all business rules.

 

I have worked on an application where data scenarios are very complex and to validate every functionality, we needed to consider multiple data combinatios. Such an application would probably not give you higher return in crowdsourced environment. We need internal testers who knows the applciation and can either create or match the data conditions to validate all scenarios.

Mukesh Sharma's picture

Not really. It all depends on what kind of testing tasks you have on hand for which you plan to use crowdsourced testing. It is very important to decide what, when and how to crowdsource in a test effort failing which the ROI can be very poor and can shake the entire team’s confidence in the model. When the right choices are made on these areas (for example, using the crowd to test for an application’s usability across devices), the ROI will most often be higher than the traditional testing engagement models. In short though, we are still at a stage where crowdsourced testing is a supplement to the traditional testing models and together the two can yield a very high ROI on the testing effort, when the effort is well planned and implemented. Co-incidentally, my collegue and I did a tutorial at Star West last year on this topic "Implementing a Successful Crowdsourced Testing Effort"

Simon Papineau's picture

My company recently did a comparative study of crowdsourced vs. outsourced testing.

What we found is that crowdsourced testing yields 4.7x the amount of bugs in 60% of the time.

That being said, not every type of project is a suitable candidate for crowdsourced testing.

As its been said by fellow respondents, a hybrid model of internal and external testing is still the best approach for maximum value and efficiency.

However, crowdsourced testing can definity contribute significant ROI while helping reduced the cost of software testing for development companies.

 

Alex Wilson's picture

I think that traditional paradigme of software testing is overrated. Nobody should keep their own QA department for volatile testing process. Crowdsourced testing looks like much more efficent solution. Application development is not steady, so you have no need to hire a certain number of full-time testers in your company. Instead of this you could hire testing professionals just at time they are really needed. one of such services we personally use in our company is Ubertesters 'Hire-testers' service - it's like we set up our own team but on-demand basis. there are lots of similar services you can find and try as well. And of course it provides a much higher ROI compared to the traditional means of testing.. with crowdtesting you don't have 'fixed costs' when you have nothing to test
©2011-2013 TechWell Corp.