This article explores whether strict adherence to software development methodologies guarantees project success. Analyzing statistical data and QA professional insights, it reveals that "by the book" application of models like Scrum is uncommon. Instead, successful projects often employ flexible, hybrid approaches, prioritizing adaptable frameworks and tailored artifact selection over rigid guidelines to achieve high quality within constraints.
Software development requires a structured approach, often referred to as a software development model. These models are often applied in accordance with international standards such as ISO/IEC 12270 [6]. The first SDLC model was a fundamental part of software engineering because it provided a structure for different software development activities.
Fundamental approaches, such as structured development methods, are utilized within the software development life cycle. A key challenge was to reduce unnecessary activities and increase productivity. The integration of supporting processes and iterative lifecycles improved productivity and product quality, leading to the development of new methodologies and frameworks aimed at further project improvement.
| 1 | R. Kneuper [2] | 2017 | Sixty years of software development life cycles history and evaluation of new software development models. | Sixty years of development model exploration without considering specific factors such as scope, budget, risk, resource, and quality. |
| 2 | R. Arora et al. [16] | 2016 | Choose the right software development model according to user needs. | The limitation of this research is that not all models are discussed as some models can be simulated using specific tools |
| 3 | M. A. Rather et al. [17] | 2016 | Different software process models are discussed and explain when the new model came into existence. | The limitation of this research is that different models are explored without a specific focus (e.g., quality, cost). |
| 4 | I. H. Sarker et al. [19] | 2015 | Survey of different development process models to choose the desired model according to the requirement. | Does not discuss the hybrid software development technique. |
| 5 | Alshamrani et al. [20] | 2015 | Comparative analysis of three software development models and discuss their pros and cons. | There is a big limitation that only three models are compared and analyzed for their strengths and weaknesses. |
The most widely-used models would be:
Methodology/Materials and Methods
In order to back up the thesis and the importance of the methodologies, we need to first identify the main common methodologies used globally. This research utilizes statistical data from a Scrum statistics article and Statista, combined with questionnaire responses from various QA groups regarding their experiences with different methodologies. This statistical data would form the basis for the research results.
In addition to a two-week questionnaire, interviews would be conducted with several senior QA professionals to gather their feedback on the collected statistics and specific projects.
Following the data collection, the results would be presented, and a discussion held with experienced senior QA professionals.
Results
Based on the research data by “Statista”:
Figure 1: Most Commonly Used Software Development Methodologies (Source: Statista)
The most commonly used is DevOps / DevSecOps. Key characteristics of this methodology include: In 2022, almost 47 percent of respondents used a DevOps or DevSecOps method for software development. DevOps is a practice that combines IT operations with software development. It aims to provide continuous delivery by shortening the systems development cycle while ensuring high-quality software. The reported reasons for choosing this method include faster time to market, security, code quality, and improved communication and collaboration among developers.
Based on the article from Echometer, about 66% of respondents are using Scrum.
Figure 2: Scrum Adoption Rates (Source: Echometer)
Based on the study shared, “State of Scrum 2017-2018”, when it comes to choosing Scrum for a project, 71 percent of executives agree that delivering value to the customer is their highest priority. Flexibility and responsiveness are a significant second priority. While improving organizational design and culture comes in last at 25 percent, good Scrum implementation is also likely to achieve that aim.
From the Scrum Alliance, the average project length has decreased to 11.6 weeks. For Agile organizations, that means projects—from a new product launch, to release are set against a standard completion timeframe of less than three months, start to finish. The average is on a downward trend, expected to continue to decrease in 2018.
The latest State of Agile Report (17th edition, 2023) indicates a shift in how organizations perceive and utilize Scrum. Adoption has declined to 63%, as many teams lean toward hybrid methodologies. In 2018, 71% of respondents selected Scrum to "deliver customer and business value", a figure that has fallen to 41% in 2023. At the same time, the goal of "accelerating time to market" has gained traction, now equally cited by 41% of respondents.
Below, the graphic clearly outlines the answers of respondents about the value Scrum methodology brings them:
 |  |
| Figure 3: Image from study: State of Scrum 2017-2018 | Figure 4: Image from study: 17th State of Agile Report (2023) |
Additionally, the study covers the percentage of Scrum in clean implementation and a mix of other methodologies.
Figure 5: Agile Approaches Image from the study: State of Scrum 2017-2018
Based on third-party statistics, in 2015, a hybrid approach combining Scrum with other methodologies was used approximately 56% of the time, while pure Scrum adoption was around 43%. Over the subsequent three years, the adoption of hybrid Scrum approaches increased to 78% by 2017, while pure Scrum usage decreased to 16%. The use of non-Scrum methodologies remained low at 6%.
Therefore, statistical data indicates that Scrum, the most prevalent SDLC methodology, is predominantly implemented in conjunction with other methodologies. Over time, strict adherence to pure Scrum has declined.
Based on the questionnaire started (a quiz conducted with a QA automation group on LinkedIn, and another QA group) to provide another view on the question:
Figure 6
The majority of respondents worked on 3-5 projects during their careers.
Some respondents have worked in the field for 10+ years.
The second question focused on projects that respondents worked on from inception to completion, aiming to identify the prevalence of well-established projects with efficient implementation:
Figure 7
The data indicates that most respondents were involved in more than one project from its initial stages. This leads to the subsequent question: “What percentage of these projects were set up to follow a team collaboration framework (Agile, Waterfall, etc.) according to ISTQB guidelines, including at least 70% of the required artifacts and rules?”
Figure 8
Based on the responses, a majority of respondents perceived the projects they were involved in from the beginning as adhering to the methodology. Specifically, 73% believed these projects strictly followed ISTQB guidelines.
Conversely, only 27% of respondents indicated that the projects they worked on did not follow ISTQB guidelines or included less than 60% of the expected artifacts.
Interestingly, when respondents were asked about projects they joined after the initial two months, the data suggested a higher rate of adherence to methodology guidelines.
Figure 9
Respondents reported that only 60% of projects they joined after the first two months had a well-established methodology aligned with ISTQB guidelines and included over 70% of the required artifacts.
The questionnaire data suggests that projects with QA involvement from the outset adhered to a majority of ISTQB-defined artifacts 73% of the time, compared to 60% for projects where QA joined later.
The previously cited statistics indicate Scrum's increasing adoption and revenue generation over the past three years. Therefore, early QA involvement may further enhance Scrum implementation and deliver greater value to clients.
Finally, video interviews were conducted with several respondents to gather additional comparative data before drawing conclusions.
Discussion
Based on the research findings, it can be confirmed that Scrum-based methodologies are the most prevalent SDLC approaches. Scrum-based methodologies have demonstrated their effectiveness in delivering projects on time and maximizing client value.
Between 2017 and 2018, pure Scrum implementation decreased, as indicated by the 'Agile Approaches' image. Simultaneously, the rise of Scrum in conjunction with other methodologies, while popular, deviates from a strict 'by the book' approach, often resembling a flexible framework where project managers select necessary artifacts.
The questionnaire, with 28 responses, suggests that strict adherence to methodology occurs in 40%-50% of projects. While the sample size is limited, these findings align with the trends observed in the larger studies.
Conclusion
In conclusion, the research findings indicate that most methodologies described in ISTQB guidelines or online documentation are not strictly followed. The reasons for this deviation are not primarily related to project start times (as suggested by questionnaire data) or client type, but rather stem from the inherent flexibility of Agile principles and the purpose of frameworks, which is to provide a rationale for artifacts and methodologies.
Therefore, to answer the question, 'Does strictly following methodologies guarantee project success?', we must define key performance indicators (KPIs) for success, such as project completion within the deadline, client satisfaction, and achievement of the product vision.
The 'State of Scrum 2017-2018' study indicates a decline in strict methodological adherence and a rise in hybrid approaches (combining artifacts from different methodologies), even though the underlying project management principles often remain rooted in Scrum. Thus, the answer to the initial question is “no.”
Project managers and team leads strategically select and combine methodologies and artifacts based on specific project and client needs, treating each artifact as a potential deliverable. This tailored approach, often termed a hybrid methodology within the professional community, appears to be the dominant strategy for achieving project success.
Regarding reference guidelines and rulebooks, the focus should shift from creating rigid, limited methodologies with fixed artifacts towards establishing a comprehensive catalog of artifacts with their associated deliverables and KPIs. This would empower future managers to select the most appropriate components for their specific projects.
Lets Hang!