Posts

Showing posts with the label numpy

Why does the convolution/cross correlation truncate to 0?

Image
Clash Royale CLAN TAG #URR8PPP Why does the convolution/cross correlation truncate to 0? Cross-correlation for uniformly sampled signals is defined as [1] Convolution for uniformly sampled signals is defined as [2] I am trying to cross-correlate or convolve two arrays of similar input data. These arrays are two similar time series voltage events that I am trying to align. However, the correlation and convolution only give back arrays full of 0. This seems strange to me, because the input arrays are observations of the same phenomena and should have some sort of overlap. I can clearly plot the input data: Whole Time Series: Zoomed in: This is the code I am running: def get_cross_corr(files): datalist = # open required files for correlation try: for f in files: fp = open(f, "rb") data = np.fromfile(fp,dtype=np.int16) datalist.append(data[0:int(len(data)/4)]) fp.close() except: print("could not ...

Change the output result of np.array

Image
Clash Royale CLAN TAG #URR8PPP Change the output result of np.array I have 3 different .txt files and each of them contains x,y,z coordinates such: I read the content of those 3 files: x,y,z inputFileList = sorted(glob.glob(inputSourceDir + '/*.txt'), key=lambda x: (int(re.sub('D', '', x)), x)) inputFileList = inputFileList[0:100] inputTotalDataList = self.numberOfInputFiles = 0 for inputFilePath in inputFileList: inputDataInFile = np.genfromtxt(inputFilePath, dtype=float, delimiter=',') # usecols= 0 baseWithExt = os.path.basename(inputFilePath) base = os.path.splitext(baseWithExt)[0] inputTotalDataList.append(inputDataInFile) self.numberOfInputFiles = self.numberOfInputFiles + 1 self.inputTotalData = np.array(inputTotalDataList) self.inputTotalData = self.inputTotalData.reshape(self.numberOfInputFiles * len(inputDataInFile), 3) print('TotalData: ', inputTotalData ) As output I get: TotalData: [[ 7.29948 -187....

Numpy not recognizing a proper dtype

Image
Clash Royale CLAN TAG #URR8PPP Numpy not recognizing a proper dtype My code is below: import numpy as np from nltk.tokenize import TweetTokenizer from nltk import pos_tag class tag_tokenizer: tokenizer = TweetTokenizer() #learn tokenizing stuff dt = np.dtype([("token", 'U16') , ("pos_tag","U5")]) def __init__(self, rawDocs): self.tagged_data = np.array([pos_tag(self.tokenizer.tokenize(rawDoc)) for rawDoc in rawDocs], dtype=self.dt) But I get the error: TypeError: a bytes-like object is required, not 'list' whenevery I try to initialize an instance of tag_tokenizer. My list is a lst of tuples of both string characters so I don't know why numpy wont let me. Do I have to create the array first, and then set the dtype, or am I just doing it wrong? By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, p...

Split sorted array into list with sublists

Image
Clash Royale CLAN TAG #URR8PPP Split sorted array into list with sublists I have a sorted array of float32 Values, I want to split this array into a list of lists containing only the same Values like this: >>> split_sorted(array) # [1., 1., 1., 2., 2., 3.] >>> [[1., 1., 1.], [2., 2.], [3.]] My current approach is this Function def split_sorted(array): split = [[array[0]]] s_index = 0 a_index = 1 while a_index < len(array): while a_index < len(array) and array[a_index] == split[s_index][0]: split[s_index].append(array[a_index]) a_index += 1 else: if a_index < len(array): s_index += 1 a_index += 1 split.append([array[a_index]]) My Question now is, is there a more Pythonic way to do this? maybe even with numpy? And is this the most performant way? Thanks a lot! What's the typical length of the input array and number ...

How to square the individual matrix value using python?

Image
Clash Royale CLAN TAG #URR8PPP How to square the individual matrix value using python? I am trying to implement the Cost function in python. Assume my data have X (loading from txt file) and my theta value is [[0] [0]] For that I have implemented as below: import pandas as pd import numpy as np import matplotlib.pyplot as plt from scipy.linalg import fractional_matrix_power load_data = pd.read_csv('C:python_programex1data1.txt',sep = ",",header = None) feature_vale = load_data[0] y = np.matrix(load_data[1]) m = len(feature_vale) #print(m) #plt.scatter(load_data[0],load_data[1]) df = pd.DataFrame(pd.Series(1,index= range(0,m))) df[1] = load_data[0] X = np.matrix(df) row_theta = np.zeros(2,dtype = int) theta = np.array([row_theta]) # Transpose the array print(theta.T) prediction = np.matmul(X,theta.T) error = (prediction-y) print(error) Output of the error I got as expect...

why scipy.spatial.ckdtree runs slower than scipy.spatial.kdtree

Image
Clash Royale CLAN TAG #URR8PPP why scipy.spatial.ckdtree runs slower than scipy.spatial.kdtree Normally,scipy.spatial.ckdtree runs much faster than scipy.spatial.kdtree. But in my case,scipy.spatial.ckdtree runs slower than scipy.spatial.kdtree. My code is as follows: import numpy as np from laspy.file import File from scipy import spatial from timeit import default_timer as timer inFile = File("Toronto_Strip_01.las") dataset = np.vstack([inFile.x, inFile.y, inFile.z]).transpose() print(dataset.shape) start=timer() tree = spatial.cKDTree(dataset) # balanced_tree = False end=timer() distance,index=tree.query(dataset[100,:],k=5) print(distance,index) print(end-start) start=timer() tree = spatial.KDTree(dataset) end=timer() dis,indices= tree.query(dataset[100,:],k=5) print(dis,indices) print(end-start) dataset.shape is (2727891, 3),dataset.max() is 4834229.32 But, in a test case, scipy.spatial.ckdtree runs much faster than scipy.spatial.kdtree,the code is as follows: import nump...

Invert y axis on actual data instead of just on the plot

Image
Clash Royale CLAN TAG #URR8PPP Invert y axis on actual data instead of just on the plot I have some locations of features that are represented in pixel coordinate space (ie. (0,0) is in the top left corner of the image, and the y axis increases downwards and the x axis increases rightwards). When I plot these locations in Matplotlib (which by default uses the positive x and positive y quadrant of a cartesian plane) I always run the command plt.gca().invert_yaxis() so that the locations of the features look right in the plot. However, I would like to apply this transformation to the image points themselves, not just in the visualization. Show your data. – John Zwinck 9 mins ago a = np.arange(9).reshape(3,3) …. np.flipud(a) …. is this what you mean? ...

What is fastest way to compute average of python array different elements with condition?

Image
Clash Royale CLAN TAG #URR8PPP What is fastest way to compute average of python array different elements with condition? I have phyton code below which executed about 2s. Equivalent code in C language executed in 31ms. #!/usr/bin/env python import time import numpy as np dd = np.random.randint(0, 20, size=(2*1000*1000)) t0= time.clock() avg_sum1=0.0 BlockOffset = 0 while BlockOffset < len(dd): if dd[BlockOffset + 1] <= 10: avg_sum1 = dd[BlockOffset + 1] * 0.1 else: avg_sum1 = dd[BlockOffset + 0] * 0.01 BlockOffset+=2 print('Avg: ' + str( avg_sum1/len(dd)/2 ) ) print('Exe time: '+ str(time.clock() - t0) ) What is fastest way to to do this with built in functions or numpy? Although the question may be on-topic here, Code Review might be more suitable for your demonstrated needs. – Jerrybibo 58 secs ago ...

What is numpy method int0?

Image
Clash Royale CLAN TAG #URR8PPP What is numpy method int0? I've seen np.int0 used for converting bounding box floating point values to int in OpenCV problems. np.int0 What exactly is np.int0 ? np.int0 I've seen np.uint8 , np.int32 , etc. I can't seem to find np.int0 in any online documentation. What kind of int does this cast arguments to? np.uint8 np.int32 np.int0 What didn't you understand from the on-line documentation and examples? – Prune Jan 19 at 22:54 @Prune: Well, there is no NumPy documentation for int0, as the questioner said. – user2357112 Jan 19 at 22:56 @user2357112: I disagree. A simple search for "numpy int0 function" brings...

Convert 1d array to lower triangular matrix

Image
Clash Royale CLAN TAG #URR8PPP Convert 1d array to lower triangular matrix I would like to convert a 1 dimensional array into a lower, zero diagonal matrix while keeping all the digits. I am aware of numpy.tril function but it replaces some of the elements with zeros. I need to expand the matrix to contain all the original digits. numpy.tril For example: [10,20,40,46,33,14,12,46,52,30,59,18,11,22,30,2,11,58,22,72,12] Should be 0 10 0 20 40 0 46 33 14 0 12 46 52 30 0 59 18 11 22 30 0 2 11 58 22 72 12 0 3 Answers 3 With the input array holding all the values as required to fill up the lower diagonal places, here's one approach with masking - masking def fill_lower_diag(a): n = int(np.sqrt(len(a)*2))+1 mask = np.tri(n,dtype=bool, k=-1) # or np.arange(n)[:,None] > np.arange(n) out = np.zeros((n,n),dtype=int) out[mask] = a return out Sample run - In [82]: a Out[82]: array([10...

numpy install error with pycharm

Image
Clash Royale CLAN TAG #URR8PPP numpy install error with pycharm Error: AttributeError: module 'pip' has no attribute 'main' Traceback (most recent call last): File "C:Program FilesJetBrainsPyCharm Community Edition 2017.3.2helperspackaging_tool.py", line 192, in main retcode = do_install(pkgs) File "C:Program FilesJetBrainsPyCharm Community Edition 2017.3.2helperspackaging_tool.py", line 109, in do_install return pip.main(['install'] + pkgs) AttributeError: module 'pip' has no attribute 'main' By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Octave(or Matlab) sub2ind function convert to python code

Image
Clash Royale CLAN TAG #URR8PPP Octave(or Matlab) sub2ind function convert to python code I implement Matlab 'strel' function to python language. I have some difficulty converting to python code. plz let me know how to convert this function. I think numpy's ravel_multi_index function equal to sub2ind. but it dose not work well. :( Octave code degrees = 30 linelen = 5 deg90 = mod (degrees, 90); if (deg90 > 45) alpha = pi * (90 - deg90) / 180; else alpha = pi * deg90 / 180; endif ray = (linelen - 1)/2; c = round (ray * cos (alpha)) + 1; r = round (ray * sin (alpha)) + 1; line = false (r, c); m = tan (alpha); x = [1:c]; y = r - fix (m .* (x - 0.5)); %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% indexes = sub2ind ([r c], y, x); %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% and My python code here. def strel_line(length,degree): import numpy as np from numpy import pi,cos,sin,tan,fix deg90 = degree % 90 if deg90 > 45: alpha = pi * (90 - deg90) / 180 else: a...

How to deal with this when I use numpy.array() to deal with 'list'?

Image
Clash Royale CLAN TAG #URR8PPP How to deal with this when I use numpy.array() to deal with 'list'? import cv2 import numpy list_pixel= list_label= for i in range(0,10): for j in range(0,10): list_pixel.append(cv2.imread("C:\Users\kimcho\Desktop\testdata\testdata_"+str(i)+"_0"+str(j)+".png",0)) list_label.append(i) j=0 list_pixel.pop(0) list_label.pop(0) list_pixel=numpy.array(list_pixel) print(list_pixel) print(list_pixel.shape) print(list_pixel[0].shape) How to deal with this when I use numpy.array() to deal with 'list'?I wanna make datasets by imitating keras.But,the datasets I made didn't satisfy me.I want it to act like keras,to return a value like this: It can return a value of(60000,28,28) But as for my datasets,it can only return like this: Only return a value of(99,)—— I got 99 pictures and I want to load their pixel into list_pixel Here is my code: Hoping anyone can help me solve this problem.Deeply tha...

np.dot 3x3 with N 1x3 arrays

Image
Clash Royale CLAN TAG #URR8PPP np.dot 3x3 with N 1x3 arrays I have an ndarray of N 1x3 arrays I'd like to perform dot multiplication with a 3x3 matrix. I can't seem to figure out an efficient way to do this, as all the multi_dot and tensordot, etc methods seem to recursively sum or multiply the results of each operation. I simply want to apply a dot multiply the same way you can apply a scalar. I can do this with a for loop or list comprehension but it is much too slow for my application. N = np.asarray([[1, 2, 3], [4, 5, 6], [7, 8, 9], ...]) m = np.asarray([[10, 20, 30], [40, 50, 60], [70, 80, 90]]) I'd like to perform something such as this but without any python loops: np.asarray([np.dot(m, a) for a in N]) so that it simply returns [m * N[0], m * N[1], m * N[2], ...] [m * N[0], m * N[1], m * N[2], ...] What's the most efficient way to do this? And is there a way to do this so that if N is just a single 1x3 matrix, it will just output the same as np.dot(m, N)? ...