Posts

Showing posts with the label bigdata

Google analytics architecture: database

Image
Clash Royale CLAN TAG #URR8PPP Google analytics architecture: database Does anyone knoe the architecture behing google analytics site? I mean especially databases. (For row data/ aggregatios) Also do they pre-aggregate the traffic data? By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Sparklyr : separate rows on 2 columns

Image
Clash Royale CLAN TAG #URR8PPP Sparklyr : separate rows on 2 columns I am using sparklyr for a project. I have a Spark Dataframe with lists in some of the columns and I'd like to separate them into multiple rows, i.e. have one value in each row, exactly like separate_rows does in dplyr . separate_rows dplyr So basically my dataframe is like this | x | y 1| [a,b] | [c,d] And I'd like to have something like this in the end : | x | y 1| a | c 2| b | d Like suggested in this post, explode is a good start, but it can do the job for only one column at once ; and if I use it twice, I will end up with 4 rows here instead of the 2 I want. In this very simple example, I could manage my way to keep only the rows that I want, but things can get a bit messier if there are more than two elements in the lists... explode Something I thought about would be to do : 1) Merge the columns x and y into a single column which would contain [[a,c] , [b,d]] x y [[a,c] , [b,d]] 2) T...