What a (work)Flow - from an idea to a report
- John Abbott
- Nov 13, 2022
- 4 min read
Regardless of what tools you are using for data analysis and reporting you always begin with a blank screen and a blinking cursor. For some, anxiety pulses through the body with each reappearance of the cursor. Where do you start, how do you start, what's the best way to get started. In this post Ill discuss some considerations for establishing a workflow for exploratory analysis and routine monitoring.
Data Gathering and Storage
Hopefully our mind is rushing with ideas about, what data you have and how it can be used, Jump Height trends related to key performance indicators (sprint speed, shot velocity, shooting accuracy, etc.) but before you get started with any analysis you need to get your data into the computer. Creating a list of data streams and the available methods to get that data into your computer is a exercise that will help improve your data hygiene and workflow organization.
Tech/Company | Test | Collection Method | Data Transfer |
|---|---|---|---|
Vald Forceplates | Jumps | Ipad/Computer | API |
Vald NordBord | Hamstring Strength | Ipad | API |
Brower Timing Gate | 40yd Sprint | Manual Read Out | Manual Entry |
GymAware | Compound Movements | Ipad | Exports |
While this list of data transfer methods is not exhaustive, but these are commonly utilized methods.. Manual Entry, grab your clip board and a pen, line the athletes up and start scratching down results. This is surely the lowest learning curve with the least upfront amount of work, but the hours of sitting at your computer poking number keys 1 by 1. The long term time commitment and risk of data entry errors makes this the least favored approach though for some assessments this is the only option.
Data Exports, find that download csv button and memorize it. So you have finished the day, you collapse down into your chair, and you start the second, third, or fourth part of your day. Data download and exports, companies like VALD save their data to the cloud and you can hop onto the website select a date range and download the data in a csv (comma separated values document). What will be absolutely CRUCIAL for your long term success and sanity is setting up a filing system for how to store these files.
API (Application Program Interface), a tool that allows for communication between two systems, for example how data gets from a company to your athlete management software. While this is the most complex method of gathering data it creates the best scenario for automating reports. If you are new to coding, tackling API Development may be an aggressive task though if you have collogues or coworkers in computer science or computer development fields it should be rather easy.
How do APIs Work
There are several types of APIs but the most common I have experienced within sport technologies are REST (Representational State Transfer) and these interfaces rely on a set of key functions: GET, PUT, DELETE, POST. With these functions and some other arguments: URL, Authorization, data type, etc. , you can pull data from companies cloud and store locally as csv's (least efficient easiest to start with) or within a database (MySQL, Redshift, and others). Databases are surely the most efficient way of storing data though requires another language to interact with SQL.
Id highly encourage that you either reach out to your company for assistance in setting up an API or to me and we can work through the process.
When using databases and API, data querying and structure is optimized by using specific data structures,. one of which is a One to Many structure. When looking below you will see three tables that you can get from the API, though not one table has all the information. Using this allows for quicker loading of data once you have it stored locally(or in a database). You can gather information from athlete info to get references. For example you can retrieve athlete ids for athletes your interested from the first table - use those ids to grab certain test ids (associated with that athlete) and lasted grab the data that you are interested from the Trial Information table. By using specific queries (filter and request of data) you can prevent yourself from loading in all historic data and filtering down which is time and memory intensive.
So Now you have your data - now what?
Data Formats you'll run into two primary formats long and wide. Depending on your software and what sort of display or functions you want to use will dictate how you manipulate between the two styles
Long Format is one of the least visually appealing though most powerful way to filter and manipulate data.. In this case you have a row for every observation, and when you add data, that data is appended to the bottom of the table.
Wide Format is a more traditional method of building tables, in particular for when considering the visual appearance. In this case you may have a rows dictating a grouping variable like an athlete but columns representing dates or variables.
The above two tables contain the same data formatted in both long and wide formats -take note that where data is not available in the long format a place holder shows up as NA.
Conclusion
So you have now started collecting data and have all of this information waiting to be explored. From this you should have started to think about how to get your data from the sources into the computer, how to structure your folders/file saving structure, and some different ways to modify tables prepare for further analysis.
From here stay posted for more consent on exploring data and developing a deep understanding of what you have.
Comments