Posts

Showing posts with the label csv

Space delimiter in CSV import to Python

Image
Clash Royale CLAN TAG #URR8PPP Space delimiter in CSV import to Python I know there are more than a few questions regarding space delimiters in CSV files. I have a CSV file that appears to be separated by a space. When importing to Python, I have tried every code out there to identify space as a delimiter. However, I keep getting error messages. For example: test_filepath = 'test_data.csv' with codecs.open(test_filepath, "r", "Shift-JIS", "ignore") as file: # import UTF8 based csv file test_df = pd.read_table( file, delim_whitespace=True ) this yields the following error: EmptyDataError: No columns to parse from file when I try this: test_filepath = 'test_data.csv' with codecs.open(test_filepath, "r", "Shift-JIS", "ignore") as file: # import UTF8 based csv file test_df = pd.read_table( file, delimiter=" " ) it gives the same error. when i try this: test_filepath = 'test_data....

Fast-csv read several files synchronously

Image
Clash Royale CLAN TAG #URR8PPP Fast-csv read several files synchronously I'm trying to read several files synchronously with fast-csv, it should looks like: read file 1 execute something while reading read file 2 execute something while reading (it must be execute after first execution's file that's why I need to do this synchronously) ... Here is my code simplified: const csv = require('fast-csv'); const PROCEDURES = [ { "name": "p1", "file": "p1.csv" }, { "name": "p2", "file": "p2.csv" }, ]; const launchProcedure = (name, file) => { try { const fs = require("fs"); const stream = fs.createReadStream(file, { encoding: 'utf8' }); console.log('launching parsing...'); stream.once('readable', () => { // ignore first line let chunk; while (null !== (chunk = stream....

printing product headers that have 0 as a value in a separate header GUI

Image
Clash Royale CLAN TAG #URR8PPP printing product headers that have 0 as a value in a separate header GUI For my code, I have to find a certain product and the values associated with it for a specific date( that are headers in the csv file) and take those numbers and manipulate them. However, there are blank values for many of the products ( because those products are not produced for those specific months), thus the reason I produced my current code, because I did not want to display adjusted values for numbers that were either blank or 0. import pandas as pd import csv import numpy as np import os from tkinter import* master = Tk() fileVar = StringVar() fileLabel = Label(master, textvariable=fileVar, font=('Consolas', 9)) fileLabel.grid(row=8, column=1) fileVar2 = StringVar() fileLabel2 = Label(master, textvariable=fileVar2, font=('Consolas', 9)) fileLabel2.grid(row=8, column=2) readfile = pd.read_csv('50.csv') filevalues= readfile.loc[readfile['Custome...

Upload CSV file to SQL server

Image
Clash Royale CLAN TAG #URR8PPP Upload CSV file to SQL server What is the best way to upload a large csv data file into SQL server using C# ? The file contains about 30,000 rows and 25 columns. csv why to use .net to upload the file while sql server itself support the import of CSV file?? – Furqan Hameedi Dec 24 '13 at 10:36 SQL Server doesn't have any facility to upload files to - you'll need to use some other means (e.g. a web service running on IIS or something like that). SQL Server can load a file from a disk it can reach - so you'll need find a way to put that file where SQL Server can read it from – marc_s Dec 24 '13 at 10:36 the requireme...

Split CSV by column value, and keep header

Image
Clash Royale CLAN TAG #URR8PPP Split CSV by column value, and keep header This has been asked many times before but I simply can't implement the solutions properly. I have a large csv named 2017-01.csv, with a date column (it's the second column in the file) and I am splitting the file by date. The original file looks like: date 2017-01-01 2017-01-01 2017-01-01 2017-01-02 2017-01-02 2017-01-02 After the split, 2017-01-01.csv looks like 2017-01-01 2017-01-01 2017-01-01 and 2017-01-02.csv looks like 2017-01-02 2017-01-02 2017-01-02 The code I am using is awk -F ',' '{print > (""$2".csv")}' 2017.csv Everything works fine but I need to keep the header row. So I tried awk -F ',' 'NR==1; NR > 1{print > (""$2".csv")}' 2017-01.csv But I still get the same results without the header row. What am I doing wrong? I read answers to many similar questions on Stackoverflow but I just can't understand w...

How to make log.csv file to log 400 .jpeg names in one row (Photoshop/JavaScript)

Image
Clash Royale CLAN TAG #URR8PPP How to make log.csv file to log 400 .jpeg names in one row (Photoshop/JavaScript) I am still new to JavaScript. But somehow managed to create following script. Usually I run this script, to batch process 400-500 images. It creates and saves 2 copies of an image in 2 different locations, while doing some specific Image Compression. //Script should be used only in Adobe Photoshop //This script will: 1. Check if canvas is 1:1, if not then scale to the correct ratio (3000px x 3000px) // 2. Check whether it has ColorProfile embedded. If not then change collor profile to sRGB. // 3. Save the resized TIFF version (100% Quality, 3000px x 3000px, sRGB, LZW compressed, Without layers) into "/Volumes/Hams Hall Workspace/Ecom_Staging/Images_Today" // 4. Save another copy as JPEG (80% Quality, 2000px x 2000px, sRGB) "into /Volumes/Hams Hall Workspace/Ecom_Staging/Jpegs_for_Hybris" if( BridgeTal...

Separate text from a csv row into multiple ones?

Image
Clash Royale CLAN TAG #URR8PPP Separate text from a csv row into multiple ones? I am having a .csv file with rows: NAME , LOCATION , IP Address . Row IP Address has many IPs in it and they are separated by a whitespace, up to 5 characters. For example: NAME LOCATION IP Address IP Address EDITED: Name,Location,IP Address Router,China,10.10.10.1 10.10.10.2 10.10.10.3 10.10.10.4 10.10.10.5 Switch,USA,192.168.1.1 192.168.1.2 192.168.1.3 and so on.. I want all the IPs to be on the separate rows under the IP Address column. This is how I want my output to look like: IP Address Name,Location,IP Address Router,China,10.10.10.1 Router,China,10.10.10.2 Router,China,10.10.10.3 Router,China,10.10.10.4 Router,China,10.10.10.5 Switch,USA,192.168.1.1 Switch,USA,192.168.1.2 Switch,USA,192.168.1.3 I have been trying to do .split() but it is giving me an error. Can anyone please help me? .split() Using csv.DictReader to read my csv files here. csv.DictReader ...

handle duplicate values separately in python

handle duplicate values separately in python I have a csv file which contains rows like below: 113,25CNFW,"test1",1ALNAU,25CNFW 113,25D5L2,"test2",1ALNAU,25D5L2 114,MCF82H,"test3",307531,MCF82H Now, I'm trying to extract 2nd and 4th column of every row by searching certain nos in the 1st column, but whenever the 1st column contains duplicates, my script logic fails as it treats duplicates as single entry. Is there any way to iterate over duplicates in python such that each duplicate value is considered as individual? import csv mycsv = "/home/test.csv" def csv_read(fobj, mid): reader = csv.reader(fobj) for row in reader: if mid in row: contid = row[1] accid = row[3] return accid,contid list = ['113','114'] for mid in list: with open(mycsv,"r") as fobj: detail = csv_read(fobj,mid) print detail[0] + "," + detail[1] Output: 1ALNAU,25CNFW 3...