Code-Sage

(Universal) Data Import Tool

09 Mar 2023

Tool

Objective

Organize and consolidate data sets from multiple sources for accounting/financial purposes.

Technical Goals/Overview

Develop import tool. Tool should be able to import varied data sets based on meta data of each data set. The meta data will be used to normalize the data sets into a common and consistent format. The normalized data sets will be consolidated into a single data set for analysis and reporting.

Summary: Technical Implementation

  1. Project Setup
    • Python Environment
    • Jupyter Notebook
    • Data Analysis Packages
  2. Iterative Data Analysis and Tool development
    • Explore, understand nuances of each data source.
    • Create meta data describing each data set
    • Refine tool and import data set.

Overall, the import tool is successfully able to import data sets from multiple sources and normalize the data sets into a common format. Additional formats can be supported by adding meta data describing the data set.

Skills Footprint:

Category Technical Specifics
general Data Science * Data Analysis
standards CSV * Spreadsheet
tools VSCode * GIT * Python * Jupyter Notebook
concepts Dataset Creation * Data Normalization * Data Processing
packages csv * json * yaml

Related posts