Unable to create hierarchical rowTag while converting Dataframe into an XML using pyspark
I'm using pyspark to convert dataframe into an xml file.
Below is my sample dataframe(df) -
Value Year Month Day
------------------------------
con 2018 10 5
tet 2017 7 14
eco 2010 12 6
The out put XML would be -
<DataValue>
<DataList>
<Value> con </Value>
<TimeStamp>
<Year> 2018 </Year>
<Month> 10 </Month>
<Day> 5 </Day>
</TimeStamp>
</DataList>
<DataList>
<Value> tet </Value>
<TimeStamp>
<Year> 2017 </Year>
<Month> 7 </Month>
<Day> 14 </Day>
</TimeStamp>
</DataList>
<DataList>
<Value> eco </Value>
<TimeStamp>
<Year> 2010 </Year>
<Month> 12 </Month>
<Day> 6 </Day>
</TimeStamp>
</DataList>
</DataValue>
I'm able to create the xml but not able to generate rowTag TimeStamp under rowtag DataList. Please assist.
python python-3.x apache-spark pyspark apache-spark-sql
asked Nov 15 '18 at 19:07
Nabarun ChakrabortiNabarun Chakraborti
235
add a comment |
I'm using pyspark to convert dataframe into an xml file.
Below is my sample dataframe(df) -
Value Year Month Day
------------------------------
con 2018 10 5
tet 2017 7 14
eco 2010 12 6
The out put XML would be -
<DataValue>
<DataList>
<Value> con </Value>
<TimeStamp>
<Year> 2018 </Year>
<Month> 10 </Month>
<Day> 5 </Day>
</TimeStamp>
</DataList>
<DataList>
<Value> tet </Value>
<TimeStamp>
<Year> 2017 </Year>
<Month> 7 </Month>
<Day> 14 </Day>
</TimeStamp>
</DataList>
<DataList>
<Value> eco </Value>
<TimeStamp>
<Year> 2010 </Year>
<Month> 12 </Month>
<Day> 6 </Day>
</TimeStamp>
</DataList>
</DataValue>
I'm able to create the xml but not able to generate rowTag TimeStamp under rowtag DataList. Please assist.
python python-3.x apache-spark pyspark apache-spark-sql
asked Nov 15 '18 at 19:07
Nabarun ChakrabortiNabarun Chakraborti
235
add a comment |
I'm using pyspark to convert dataframe into an xml file.
Below is my sample dataframe(df) -
Value Year Month Day
------------------------------
con 2018 10 5
tet 2017 7 14
eco 2010 12 6
The out put XML would be -
<DataValue>
<DataList>
<Value> con </Value>
<TimeStamp>
<Year> 2018 </Year>
<Month> 10 </Month>
<Day> 5 </Day>
</TimeStamp>
</DataList>
<DataList>
<Value> tet </Value>
<TimeStamp>
<Year> 2017 </Year>
<Month> 7 </Month>
<Day> 14 </Day>
</TimeStamp>
</DataList>
<DataList>
<Value> eco </Value>
<TimeStamp>
<Year> 2010 </Year>
<Month> 12 </Month>
<Day> 6 </Day>
</TimeStamp>
</DataList>
</DataValue>
I'm able to create the xml but not able to generate rowTag TimeStamp under rowtag DataList. Please assist.
python python-3.x apache-spark pyspark apache-spark-sql
asked Nov 15 '18 at 19:07
Nabarun ChakrabortiNabarun Chakraborti
235
I'm using pyspark to convert dataframe into an xml file.
Below is my sample dataframe(df) -
Value Year Month Day
------------------------------
con 2018 10 5
tet 2017 7 14
eco 2010 12 6
The out put XML would be -
<DataValue>
<DataList>
<Value> con </Value>
<TimeStamp>
<Year> 2018 </Year>
<Month> 10 </Month>
<Day> 5 </Day>
</TimeStamp>
</DataList>
<DataList>
<Value> tet </Value>
<TimeStamp>
<Year> 2017 </Year>
<Month> 7 </Month>
<Day> 14 </Day>
</TimeStamp>
</DataList>
<DataList>
<Value> eco </Value>
<TimeStamp>
<Year> 2010 </Year>
<Month> 12 </Month>
<Day> 6 </Day>
</TimeStamp>
</DataList>
</DataValue>
I'm able to create the xml but not able to generate rowTag TimeStamp under rowtag DataList. Please assist.
python python-3.x apache-spark pyspark apache-spark-sql
python python-3.x apache-spark pyspark apache-spark-sql
asked Nov 15 '18 at 19:07
Nabarun ChakrabortiNabarun Chakraborti
235
asked Nov 15 '18 at 19:07
Nabarun ChakrabortiNabarun Chakraborti
235
edited Nov 17 '18 at 15:50
Nabarun Chakraborti
asked Nov 15 '18 at 19:07
Nabarun ChakrabortiNabarun Chakraborti
235
asked Nov 15 '18 at 19:07
Nabarun ChakrabortiNabarun Chakraborti
235
asked Nov 15 '18 at 19:07
Nabarun ChakrabortiNabarun Chakraborti
235
235
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
I'm able to solve my above problem. Please note we can only create a root tag and row tags. But there is no direct approach to create subtags inside rowtags. If you want to create any additional sub tag under row tag you have to create a structure and store in your dataframe column.
In my above case I want Timestamp sub tag which contains Year Month and Day. So, I've defined a schema for those 3 fields and create an Array with that schema and add it in my dataframe. Finally when I converted that specific column (which coiontains the [yyyy, mm, dd] array) into an XML it appears as a sub tag. Hope this idea will help.
answered Feb 1 at 12:27
Nabarun ChakrabortiNabarun Chakraborti
235
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53326365%2funable-to-create-hierarchical-rowtag-while-converting-dataframe-into-an-xml-usin%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
I'm able to solve my above problem. Please note we can only create a root tag and row tags. But there is no direct approach to create subtags inside rowtags. If you want to create any additional sub tag under row tag you have to create a structure and store in your dataframe column.
In my above case I want Timestamp sub tag which contains Year Month and Day. So, I've defined a schema for those 3 fields and create an Array with that schema and add it in my dataframe. Finally when I converted that specific column (which coiontains the [yyyy, mm, dd] array) into an XML it appears as a sub tag. Hope this idea will help.
answered Feb 1 at 12:27
Nabarun ChakrabortiNabarun Chakraborti
235
add a comment |
I'm able to solve my above problem. Please note we can only create a root tag and row tags. But there is no direct approach to create subtags inside rowtags. If you want to create any additional sub tag under row tag you have to create a structure and store in your dataframe column.
In my above case I want Timestamp sub tag which contains Year Month and Day. So, I've defined a schema for those 3 fields and create an Array with that schema and add it in my dataframe. Finally when I converted that specific column (which coiontains the [yyyy, mm, dd] array) into an XML it appears as a sub tag. Hope this idea will help.
answered Feb 1 at 12:27
Nabarun ChakrabortiNabarun Chakraborti
235
add a comment |
I'm able to solve my above problem. Please note we can only create a root tag and row tags. But there is no direct approach to create subtags inside rowtags. If you want to create any additional sub tag under row tag you have to create a structure and store in your dataframe column.
In my above case I want Timestamp sub tag which contains Year Month and Day. So, I've defined a schema for those 3 fields and create an Array with that schema and add it in my dataframe. Finally when I converted that specific column (which coiontains the [yyyy, mm, dd] array) into an XML it appears as a sub tag. Hope this idea will help.
answered Feb 1 at 12:27
Nabarun ChakrabortiNabarun Chakraborti
235
I'm able to solve my above problem. Please note we can only create a root tag and row tags. But there is no direct approach to create subtags inside rowtags. If you want to create any additional sub tag under row tag you have to create a structure and store in your dataframe column.
In my above case I want Timestamp sub tag which contains Year Month and Day. So, I've defined a schema for those 3 fields and create an Array with that schema and add it in my dataframe. Finally when I converted that specific column (which coiontains the [yyyy, mm, dd] array) into an XML it appears as a sub tag. Hope this idea will help.
answered Feb 1 at 12:27
Nabarun ChakrabortiNabarun Chakraborti
235
edited Feb 1 at 13:11
answered Feb 1 at 12:27
Nabarun ChakrabortiNabarun Chakraborti
235
answered Feb 1 at 12:27
Nabarun ChakrabortiNabarun Chakraborti
235
answered Feb 1 at 12:27
Nabarun ChakrabortiNabarun Chakraborti
235
235
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53326365%2funable-to-create-hierarchical-rowtag-while-converting-dataframe-into-an-xml-usin%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
