Articles

Extract, transform, load Testing the Extract, Transform, and Load Process in Data Warehouses

Pulling data from a source system and putting it into a data warehouse is a process commonly known as extract, transform, and load, or ETL. Testing the process can be a chore—you need to be sure all appropriate data is extracted, that it is transformed correctly to match the data warehouse schema, and that it's all imported. Instead of testing the ETL process as a black box, you can pull it apart, testing each piece in isolation.

Matthew Heusser's picture Matthew Heusser
Data—binary code The Value of Making Your Data Sources Reusable across Test Automation Tools

Many automation tools have a mechanism for storing data used in their test scripts. Typically, the specifics of this mechanism is different across tools, making it difficult to use this data outside the tool itself. Using an external, reusable data source allows organizations to avoid the cost of migrating or duplicating existing data, thereby future-proofing their frameworks.

Paul Grizzaffi's picture Paul Grizzaffi
Big data Big Data’s Relationship with Business Intelligence and Data Warehousing

You’ve probably heard the buzz about big data and business intelligence data warehouses. Both deal with collecting information for analysis, but how are they different? When should you use one or the other? This article explains these two data solutions in a user-friendly way with real-world examples.

Nels Hoenig's picture Nels Hoenig
Business Intelligence and Data Quality Business Intelligence and Data Quality

Business analysis is only as good as the quality of the data. If the testing process is weak and the data quality and data integrity tests are suspect, then the business could be at risk. Learn how to get the most out of your data, warehouse, and business intelligence testing.

Paul Fratellone's picture Paul Fratellone

Better Software Magazine Articles

Attacking Quality Issues in Data Warehousing

To fully detect, isolate, and resolve quality issues in a traditional, large-scale data warehouse requires that several approaches be used together. Wayne identifies types of data quality issues and then illustrates how to best attack and resolve those pesky issues.

Wayne Yaddow's picture Wayne Yaddow

Interviews

Matt Coatney discusses the misconceptions behind big data Bold Questions over Big Data: An Interview with Matt Coatney

In this interview, Matt Coatney discusses the importance of asking bold questions, the big misconceptions behind big data, the best way to start your approach to big data, and his vision for a future where technology and big data make the world a better place.

Cameron Philipp-Edmonds's picture Cameron Philipp-Edmonds
Manish Arora discusses big data Demystifying Big Data: An Interview with Manish Arora

In this interview, Manish Arora demystifies big data by covering some of the biggest misperceptions and pain points held by businesses and SMEs. Arora also talks about his recent article featured on LinkedIn and why it's important to put good teams and technology into proper perspective.

Cameron Philipp-Edmonds's picture Cameron Philipp-Edmonds
Mike Trites talks about improving the quality of your metrics Improving the Efficiency of Your Metrics: An Interview with Mike Trites
Podcast

In this interview, Mike Trites, a senior test consultant, talks about his upcoming presentation at STAREAST 2014, the future of metrics, the importance of improving the efficiency of your metrics, and even an interesting take on the old phrase that numbers never lie.

Cameron Philipp-Edmonds's picture Cameron Philipp-Edmonds

Conference Presentations

STAREAST Big Data Migration to the Cloud: Testing Challenges and Strategies
Slideshow

Moving to the cloud is no longer a question of if, but when. Most corporations are either underway in their cloud adoption or have it on their radar. Typically the move from on-premise to cloud is a few hops and different types of data, such as SQL or some version of a file. Couple this with data transformations and it poses a challenge to testing and QA. How do you validate at each hop? Is it required to validate contents between source and destination? Can this testing be automated? Do we build a tool to automate these steps or purchase one? In this session, Sanjay Srinivas will discuss a real-world example showing how one QA team faced these challenges. He will weigh the options of building versus buying a tool and outline what you need to help solve your big-data needs.

[video:https://youtu.be/LDN4EMFB--8 width:300 height:200 align:right]

Sanjay Srinivas
STAREAST Data Curation: Refine and Shine
Slideshow

We now live in a world where data is generated with every action taken. From buying groceries to walking the dog, we're generating data all the time, everywhere. Companies are starting to undertake harnessing that data efficiently for business cases, and that requires developing a process around data curation. This process must determine which data to curate, how to maintain curated data, and when to delete stale data. A robust data curation process agreed upon by stakeholders is essential to mining data effectively if you want to strike gold. Michael Hobbs will walk through the lifecycle of data curation, illuminating key insights and experiences gained from Dell EMC's data curation process. Most importantly, he'll correlate the importance of good data curation with successful business cases. You'll learn the fundamental requirements that enable businesses to convert a data swamp into a data gold mine.

Michael Hobbs
BSE Testing Machine Data Is EVERYWHERE: Use It for Testing
Slideshow

As more applications are hosted on servers, they produce immense quantities of logging data. Quality engineers should verify that apps are producing log data that is existent, correct, consumable, and complete. Otherwise, apps in production are not easily monitored, have issues that are...

Tom Chavez
BSE Testing Leverage Streaming Data in a Microservices Ecosystem
Slideshow

Imagine a world where operational data is continuously flowing from applications and devices at an extremely high rate. Now imagine services intercepting this data and analyzing it real time. Sounds futuristic? It's not—it's here today. Mark Richards describes what streaming architecture...

Mark Richards

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.