The text below is selected, press Ctrl+C to copy to your clipboard. (⌘+C on Mac) No line numbers will be copied.
Guest
Python Help: Open tab delimited CSV and saving list of dicts to JSON file using json.dump() or pandas dataframe and to_json() - Working examples
By Guest on 11th November 2022 10:24:29 PM | Syntax: PYTHON | Views: 296


New Paste New paste | Download Paste Download | Toggle Line Numbers Show/Hide line no. | Copy Paste Copy text to clipboard
  1. Python: Opening tab delimited csv file and loading the data to a list of dicts
  2.  
  3. i = 0
  4. rows = []
  5. with open('data/AAPLUSUSD_H4.csv', 'r', encoding='utf-8') as f:
  6.     for row in f:
  7.         print(row.strip().split('\t'))
  8.         rows.append(row.strip().split('\t'))
  9.         i += 1
  10.         if i > 3:
  11.             break
  12.  
  13. rows_dict = [dict(zip(rows[0], row)) for row in rows[1:]]
  14.  
  15. print(rows_dict)
  16.  
  17. >>>['Time', 'Open', 'High', 'Low', 'Close', 'Volume']
  18. ['2017-02-01 20:00:00', '127.07', '128.43', '127.06', '128.163', '90']
  19. ['2017-02-02 00:00:00', '128.16', '130.112', '127.851', '129.983', '240']
  20. ['2017-02-02 04:00:00', '129.984', '130.48', '128.78', '128.785', '60']
  21. [{'Time': '2017-02-01 20:00:00', 'Open': '127.07', 'High': '128.43', 'Low': '127.06', 'Close': '128.163', 'Volume': '90'}, {'Time': '2017-02-02 00:00:00', 'Open': '128.16', 'High': '130.112', 'Low': '127.851', 'Close': '129.983', 'Volume': '240'}, {'Time': '2017-02-02 04:00:00', 'Open': '129.984', 'High': '130.48', 'Low': '128.78', 'Close': '128.785', 'Volume': '60'}]
  22.  
  23.  
  24. 1) Saving list of dicts to JSON file using json.dump()
  25.  
  26. import json
  27. with open('data/test.json', 'w', encoding='utf-8') as f:
  28.     json.dump(rows_dict, f, ensure_ascii=False, indent=4)
  29.  
  30.  
  31. 2) Saving list of dicts to json file using pandas dataframe and to_json()
  32.  
  33. import pandas as pd
  34. df = pd.DataFrame(data=rows[1:], columns=rows[0])
  35. json_rows = df.to_json('data/test_pandas.json', orient='records')
  36.  
  37. ### Note: for the df.to_json method, the orient='records' needs to be used to get the output similar to the one listed in #1.
  38.  
  39. There are different ways to format the JSON string. You’ll need to set the orient to your desired format. Here are the options:
  40.  
  41. split
  42. records
  43. index
  44. values
  45. table
  46. columns (the default format)
  47.  
  48. For more info: https://datatofish.com/export-pandas-dataframe-json/
  49.  
  50. The json output file will look like this: test.json and test_pandas.json will both contain this same data.
  51.  
  52. [
  53.     {
  54.         "Time": "2017-02-01 20:00:00",
  55.         "Open": "127.07",
  56.         "High": "128.43",
  57.         "Low": "127.06",
  58.         "Close": "128.163",
  59.         "Volume": "90"
  60.     },
  61.     {
  62.         "Time": "2017-02-02 00:00:00",
  63.         "Open": "128.16",
  64.         "High": "130.112",
  65.         "Low": "127.851",
  66.         "Close": "129.983",
  67.         "Volume": "240"
  68.     },
  69.     {
  70.         "Time": "2017-02-02 04:00:00",
  71.         "Open": "129.984",
  72.         "High": "130.48",
  73.         "Low": "128.78",
  74.         "Close": "128.785",
  75.         "Volume": "60"
  76.     }
  77. ]
  78.  
  79.  
  80.  
  81. How To Use JSON In Python
  82. https://www.youtube.com/watch?v=-51jxlQaxyA
















Python software and documentation are licensed under the PSF License Agreement.
Starting with Python 3.8.6, examples, recipes, and other code in the documentation are dual licensed under the PSF License Agreement and the Zero-Clause BSD license.
Some software incorporated into Python is under different licenses. The licenses are listed with code falling under that license. See Licenses and Acknowledgements for Incorporated Software for an incomplete list of these licenses.

Python and it's documentation is:
Copyright © 2001-2022 Python Software Foundation. All rights reserved.
Copyright © 2000 BeOpen.com. All rights reserved.
Copyright © 1995-2000 Corporation for National Research Initiatives. All rights reserved.
Copyright © 1991-1995 Stichting Mathematisch Centrum. All rights reserved.

See History and License for complete license and permissions information:
https://docs.python.org/3/license.html#psf-license
  • Recent Pastes