Resources

A "Business Data Pipeline" (BDP) is a way to describe data moving from a system of record like Workday or AIM to another system like Tableau or the locker rental app. A good BDP will connect all of the following activities, where BYU:

1. Collects and assembles data
2. Securely transfers and stores data
3. Accounts for and provides access to data
4. Uses and reports data

See here for Developer Resources

DATA GOVERNANCE

As BYU collects and assembles data, the data steward determines what datasets will be shared in the BDP. These datasets are individually classified by the data steward as either public, internal, confidential, or restricted. To receive access to a dataset in the BDP, the requestor must have a data sharing agreement approved by the data steward. Data stewards are appointed by data trustees (Vice Presidents).

VALUE-ADDED BY BDP

When the data steward requests a dataset be included in the BDP, OIT provides expert data modeling to ensure high data quality and provides secure dataset storage. Pre-building datasets simplifies the data catalog and request process. The review and approval of the data sharing agreements are easier with pre-built datasets than with ad hoc data fields. Pre-built datasets also increases overall data quality.

DATA ANALYTICS WITH BDP

Sometimes, pre-built datasets in BDPs need to be blended in order to facilitate data analytics and reporting. This is especially true when different domains must be blended like student data, human resource data, and financial data. The blending of pre-built datasets from different BDPs is done using the same expert data modeling and secure dataset storage. Blending also uses the same data governance steps.

TECHNOLOGY INVOLVED IN A BDP

Leading off-the-shelf software is used in a BDP. To model datasets and securely transfer and store them, OIT uses Informatica and Amazon Web Services. To account for and provide access to the datasets, OIT uses Dremio and Tyk. OIT has teams of employees trained in these software titles and has good vendor relationships.

Frequently Asked Questions

Download PowerPoint Slides on this topic.

The "front door" to a Business Data Pipeline (BDP) is called a data workgroup. A data workgroup is typically made up of representatives from a campus unit (those needing data) and OIT's data team. Through these workgroups, OIT helps the campus unit with three types of requests:

1. Submitting a data sharing agreement to the data steward for an existing dataset
2. Requesting a new dataset to be built (with the approval of the data steward)
3. Requesting a new BDP

Good data quality will maximize the following:

1. Accuracy
2. Completeness
3. Consistency
4. Validity
5. Timeliness

DATA ACCURACY

Data that reflect the real world has good data accuracy. Example: A student moves to a new apartment and the new address is reflected in the student information system.

DATA COMPLETENESS

Good data completeness significantly reduces missing information. Example: A student's phone number includes all ten digits.

DATA CONSISTENCY

Good data consistency is when source data and delivered data match. Example: Information in the financial system moves through a business data pipeline to another application. The data in both systems match.

DATA VALIDITY

Information that fits the defined data parameters is considered valid. Example: A data field is defined as a numbers only field and the data in the field are only numbers.

DATA TIMELINESS

Data snapshots that refresh within the defined frequency are considered timely. Example: A data snapshot has a daily refresh frequency and it is refreshed at 4 a.m. every morning.

Download PowerPoint Slides on this topic.

Disaster Recovery Plan Current State

Data Steward and Custodian Assignments

We are constantly working to update and publish our portfolio standards. Check in periodically for updates.