Sparklyr : separate rows on 2 columns
Clash Royale CLAN TAG #URR8PPP Sparklyr : separate rows on 2 columns I am using sparklyr for a project. I have a Spark Dataframe with lists in some of the columns and I'd like to separate them into multiple rows, i.e. have one value in each row, exactly like separate_rows does in dplyr . separate_rows dplyr So basically my dataframe is like this | x | y 1| [a,b] | [c,d] And I'd like to have something like this in the end : | x | y 1| a | c 2| b | d Like suggested in this post, explode is a good start, but it can do the job for only one column at once ; and if I use it twice, I will end up with 4 rows here instead of the 2 I want. In this very simple example, I could manage my way to keep only the rows that I want, but things can get a bit messier if there are more than two elements in the lists... explode Something I thought about would be to do : 1) Merge the columns x and y into a single column which would contain [[a,c] , [b,d]] x y [[a,c] , [b,d]] 2) T...