JSON Object to CSV Question

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

JSON Object to CSV Question

Steve Hayes
On Wed, 17 Jun 2015 17:49:35 -0400, Saran Ahluwalia
<ahlusar.ahluwalia at gmail.com> wrote:

>Good Evening Everyone:
>
>I would like to have this JSON object written out to a CSV file so that the

You've already said that in another thread, and got several answers.
What are you? Some kind of troll?


--
Steve Hayes from Tshwane, South Africa
Web:  http://www.khanya.org.za/stevesig.htm
Blog: http://khanya.wordpress.com
E-mail - see web page, or parse: shayes at dunelm full stop org full stop uk

Reply | Threaded
Open this post in threaded view
|

JSON Object to CSV Question

Saran Ahluwalia
 Steve,

No I am not a troll. I am attempting to clarify an evolving problem. If you read the most recent thread that I just posted it states far more information - in my humble opinion.
Peter was of great assistance. The data becomes more convoluted; as a fairly new programmer I am looking for best practice.

Finally, although this is virtual space, it is important to maintain etiquette. I would appreciate it if you would maintain civility and neutrality in discourse.

Saran

Sent from my iPhone

> On Jun 19, 2015, at 1:21 AM, Steve Hayes <hayesstw at telkomsa.net> wrote:
>
> On Wed, 17 Jun 2015 17:49:35 -0400, Saran Ahluwalia
> <ahlusar.ahluwalia at gmail.com> wrote:
>
>> Good Evening Everyone:
>>
>> I would like to have this JSON object written out to a CSV file so that the
>
> You've already said that in another thread, and got several answers.
> What are you? Some kind of troll?
>
>
> --
> Steve Hayes from Tshwane, South Africa
> Web:  http://www.khanya.org.za/stevesig.htm
> Blog: http://khanya.wordpress.com
> E-mail - see web page, or parse: shayes at dunelm full stop org full stop uk
> --
> https://mail.python.org/mailman/listinfo/python-list

Reply | Threaded
Open this post in threaded view
|

JSON Object to CSV Question

Cecil Westerhof
In reply to this post by Steve Hayes
Top posting is frowned upon. Could you put your reply under where you
reply on next time?

On Friday 19 Jun 2015 11:50 CEST, Saran Ahluwalia wrote:

> No I am not a troll. I am attempting to clarify an evolving problem.
> If you read the most recent thread that I just posted it states far
> more information - in my humble opinion. Peter was of great

You should put more thought about how your postings will look for the
receiver. I also saw those mails and was wondering why you kept
posting the same question. You was not, but you seemed to. Besides
that it is annoying for the users of the list, you lower the change of
getting a response. When I think I saw the question already, I get
annoyed, do not read it, so I will not respond.

--
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

Reply | Threaded
Open this post in threaded view
|

JSON Object to CSV Question

Saran Ahluwalia
Thank you for clarifying. I will be more cognizant to follow this protocol.

Sent from my iPhone

> On Jun 19, 2015, at 6:07 AM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>
> Top posting is frowned upon. Could you put your reply under where you
> reply on next time?
>
>> On Friday 19 Jun 2015 11:50 CEST, Saran Ahluwalia wrote:
>>
>> No I am not a troll. I am attempting to clarify an evolving problem.
>> If you read the most recent thread that I just posted it states far
>> more information - in my humble opinion. Peter was of great
>
> You should put more thought about how your postings will look for the
> receiver. I also saw those mails and was wondering why you kept
> posting the same question. You was not, but you seemed to. Besides
> that it is annoying for the users of the list, you lower the change of
> getting a response. When I think I saw the question already, I get
> annoyed, do not read it, so I will not respond.
>
> --
> Cecil Westerhof
> Senior Software Engineer
> LinkedIn: http://www.linkedin.com/in/cecilwesterhof
> --
> https://mail.python.org/mailman/listinfo/python-list

Reply | Threaded
Open this post in threaded view
|

JSON Object to CSV Question

Saran Ahluwalia
In reply to this post by Cecil Westerhof
On Friday, June 19, 2015 at 6:16:40 AM UTC-4, Cecil Westerhof wrote:

> Top posting is frowned upon. Could you put your reply under where you
> reply on next time?
>
> On Friday 19 Jun 2015 11:50 CEST, Saran Ahluwalia wrote:
>
> > No I am not a troll. I am attempting to clarify an evolving problem.
> > If you read the most recent thread that I just posted it states far
> > more information - in my humble opinion. Peter was of great
>
> You should put more thought about how your postings will look for the
> receiver. I also saw those mails and was wondering why you kept
> posting the same question. You was not, but you seemed to. Besides
> that it is annoying for the users of the list, you lower the change of
> getting a response. When I think I saw the question already, I get
> annoyed, do not read it, so I will not respond.
>
> --
> Cecil Westerhof
> Senior Software Engineer
> LinkedIn: http://www.linkedin.com/in/cecilwesterhof

On another note, are there any suggestions for how to address this problem. I should elaborate that I have recently switched careers in to development (formally an educator). I am seeking guidance for best practice. Thank you for your help everyone.

Reply | Threaded
Open this post in threaded view
|

JSON Object to CSV Question

Saran Ahluwalia
On Friday, June 19, 2015 at 8:47:25 AM UTC-4, Sahlusar wrote:

> On Friday, June 19, 2015 at 6:16:40 AM UTC-4, Cecil Westerhof wrote:
> > Top posting is frowned upon. Could you put your reply under where you
> > reply on next time?
> >
> > On Friday 19 Jun 2015 11:50 CEST, Saran Ahluwalia wrote:
> >
> > > No I am not a troll. I am attempting to clarify an evolving problem.
> > > If you read the most recent thread that I just posted it states far
> > > more information - in my humble opinion. Peter was of great
> >
> > You should put more thought about how your postings will look for the
> > receiver. I also saw those mails and was wondering why you kept
> > posting the same question. You was not, but you seemed to. Besides
> > that it is annoying for the users of the list, you lower the change of
> > getting a response. When I think I saw the question already, I get
> > annoyed, do not read it, so I will not respond.
> >
> > --
> > Cecil Westerhof
> > Senior Software Engineer
> > LinkedIn: http://www.linkedin.com/in/cecilwesterhof
>
> On another note, are there any suggestions for how to address this problem. I should elaborate that I have recently switched careers in to development (formally an educator). I am seeking guidance for best practice. Thank you for your help everyone.

Here is a more (I hope focused question):

http://stackoverflow.com/questions/30938462/using-python-to-convert-json-objects-in-file-to-csv-2-parts.


Reply | Threaded
Open this post in threaded view
|

JSON Object to CSV Question

Steven D'Aprano-8
In reply to this post by Steve Hayes
On Fri, 19 Jun 2015 07:50 pm, Saran Ahluwalia wrote:

> If you read the most recent thread that I just posted it states far more
> information

The problem is, you are posting *too much* of the *wrong* information.

The exception you are getting appears to be a simple one: you are getting
the exception

ValueError: too many values to unpack


but since you don't show the traceback, only the error message, we have no
idea *where* that exception is happening. It could be a bug in your code,
in the JSON library, or who knows where.

Python provides lots of useful debugging information in the traceback, but
you are ignoring it. You should look at the complete traceback, everything
from the initial line:

    Traceback

to the end of the error message, not just the error message itself. That
will show you the series of function calls that lead to the error, and
usually also the actual line of code that caused the problem. That alone
might be enough to solve the problem. If not, it is one extra clue. Either
way, when asking for help, you should always post the complete traceback
(unless it is hundreds of lines long, in which case, ask first).


Unfortunately you have dumped over 100 lines of code in our lap, from two
different files. To solve this problem for you, we would have to copy your
code, install the libraries you use, somehow get a copy of your XML file:

C:\\Users\\wynsa2\\Desktop\\Python Folder\\PCSU\\Trial2_PCSU\\2-Response.xml

(don't ask me how we can possibly do that!), run your code, see what the
full traceback says, and debug the code for you. That is simply not going
to happen, not unless you pay someone.

The most critical skill a programmer can have is to learn how to focus on
what is essential when troubleshooting, and not get lost in a sea of
irrelevant code. The way to do that is to narrow the problem down to the
smallest possible code sample which demonstrates the same error, and see
whether that shows you the solution. If it does, you have your solution. If
not, you can gradually add more complexity until you have your answer.

You can read more about this here:

http://sscce.org/

Although it is written for Java, the principles apply equally to Python.

Here is the minimum code needed to give your same error:

py> x, y, z = (1, 2, 3, 4)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
ValueError: too many values to unpack


Here I am trying to assign four values into three variables. The solution is
obvious:

py> a, x, y, z = (1, 2, 3, 4)

will work. Does that help you see the solution you need? If not, we will
need a SSCCE to work with.


Some other things to consider.

You can avoid all those doubled backslashes by using forward slashes.
Windows accepts both:

C:/Users/wynsa2/Desktop/Python Folder/PCSU/Trial2_PCSU/2-Response.xml

which makes for much neater and easier file names.

Also, it doesn't help to ask multiple unrelated questions in the same post.
If they are directly related, that is okay, but you are asking (at least)
two questions:

- how to fix the ValueError exception;

- something about "I am wondering if there is a function or methodology that
would allow me to remove such nested keys and reassign the new keys to the
outer key..."

The second question is very vague, and vague questions get vague answers (or
no answers at all.) Yes, there is such a methodology. It is
called "programming".

Again, simplify what you are trying to do. Show us a *minimal* example of
the data you start with, and the result you hope to end up with. I'm afraid
I don't understand your question at all, simplifying it and showing an
example will help a lot.

You should post these in separate threads. People interested in one thread
may not be interested in the other. By posting them together, you only get
responses from people interested in both.


--
Steven


Reply | Threaded
Open this post in threaded view
|

JSON Object to CSV Question

Saran Ahluwalia
Hi Steven,

Okay, I understand now how convoluted this sounds. Here is a more focused
issue (I hope) referencing the following sample JSON object within a file:


   1. "PAC": {
   2. "Account": [{
   3. "PC": "0",
   4. "CMC": "0",
   5. "WC": "0",
   6. "DLA": "0",
   7. "CN": null,
   8. "FC": {
   9. "Int32": ["0",
   10. "0",
   11. "0",
   12. "0",
   13. "0"]
   14. },
   15. "F": {
   16. "Description": null,
   17. "Code": "0"
   18. }
   19.

The rationale for the following: the out put will need to be a CSV. All of
the keys should be headers in the CSV. The values are all corresponding
with the headers. However, Int32 should be deleted and the key reassigned
to FC. Account should be deleted and reassigned to PAC.

Essentially, where all of the instances where you have a key with a list of
dictionary, that outer key should be deleted and reassigned to the
outermost key.

Finally (in this example) the "0"s should be all concatenated in the output
CSV so accessing the lists will need to be addressed first. I know that the
following:

def hook(obj):
    return obj
def flatten(obj):
    for k, v in obj:
        if isinstance(v, list):
            yield from flatten(v)
        else:
            yield k, v
if __name__ == "__main__":
    with open("somefileneame.json") as f:
        data = json.load(f, object_pairs_hook=hook)
    pairs = list(flatten(data))
    writer = csv.writer(sys.stdout)
    header = writer.writerow([k for k, v in pairs])
    row = writer.writerow([v for k, v in pairs]) #writer.writerows for
any other iterable object

is not the best strategy. The list objects cannot be accessed (hence, the
"ValueError: too many values to unpack") is caught at the function
json.loads(). I am wondering how to add the exception when 1) when one
encounters the list and 2) how to delete and re-asssign the keys (as
described above).

Thank you for your guidance.

On Fri, Jun 19, 2015 at 10:47 AM, Steven D'Aprano <steve at pearwood.info>
wrote:

> On Fri, 19 Jun 2015 07:50 pm, Saran Ahluwalia wrote:
>
> > If you read the most recent thread that I just posted it states far more
> > information
>
> The problem is, you are posting *too much* of the *wrong* information.
>
> The exception you are getting appears to be a simple one: you are getting
> the exception
>
> ValueError: too many values to unpack
>
>
> but since you don't show the traceback, only the error message, we have no
> idea *where* that exception is happening. It could be a bug in your code,
> in the JSON library, or who knows where.
>
> Python provides lots of useful debugging information in the traceback, but
> you are ignoring it. You should look at the complete traceback, everything
> from the initial line:
>
>     Traceback
>
> to the end of the error message, not just the error message itself. That
> will show you the series of function calls that lead to the error, and
> usually also the actual line of code that caused the problem. That alone
> might be enough to solve the problem. If not, it is one extra clue. Either
> way, when asking for help, you should always post the complete traceback
> (unless it is hundreds of lines long, in which case, ask first).
>
>
> Unfortunately you have dumped over 100 lines of code in our lap, from two
> different files. To solve this problem for you, we would have to copy your
> code, install the libraries you use, somehow get a copy of your XML file:
>
> C:\\Users\\wynsa2\\Desktop\\Python
> Folder\\PCSU\\Trial2_PCSU\\2-Response.xml
>
> (don't ask me how we can possibly do that!), run your code, see what the
> full traceback says, and debug the code for you. That is simply not going
> to happen, not unless you pay someone.
>
> The most critical skill a programmer can have is to learn how to focus on
> what is essential when troubleshooting, and not get lost in a sea of
> irrelevant code. The way to do that is to narrow the problem down to the
> smallest possible code sample which demonstrates the same error, and see
> whether that shows you the solution. If it does, you have your solution. If
> not, you can gradually add more complexity until you have your answer.
>
> You can read more about this here:
>
> http://sscce.org/
>
> Although it is written for Java, the principles apply equally to Python.
>
> Here is the minimum code needed to give your same error:
>
> py> x, y, z = (1, 2, 3, 4)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> ValueError: too many values to unpack
>
>
> Here I am trying to assign four values into three variables. The solution
> is
> obvious:
>
> py> a, x, y, z = (1, 2, 3, 4)
>
> will work. Does that help you see the solution you need? If not, we will
> need a SSCCE to work with.
>
>
> Some other things to consider.
>
> You can avoid all those doubled backslashes by using forward slashes.
> Windows accepts both:
>
> C:/Users/wynsa2/Desktop/Python Folder/PCSU/Trial2_PCSU/2-Response.xml
>
> which makes for much neater and easier file names.
>
> Also, it doesn't help to ask multiple unrelated questions in the same post.
> If they are directly related, that is okay, but you are asking (at least)
> two questions:
>
> - how to fix the ValueError exception;
>
> - something about "I am wondering if there is a function or methodology
> that
> would allow me to remove such nested keys and reassign the new keys to the
> outer key..."
>
> The second question is very vague, and vague questions get vague answers
> (or
> no answers at all.) Yes, there is such a methodology. It is
> called "programming".
>
> Again, simplify what you are trying to do. Show us a *minimal* example of
> the data you start with, and the result you hope to end up with. I'm afraid
> I don't understand your question at all, simplifying it and showing an
> example will help a lot.
>
> You should post these in separate threads. People interested in one thread
> may not be interested in the other. By posting them together, you only get
> responses from people interested in both.
>
>
> --
> Steven
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150619/90544184/attachment.html>

Reply | Threaded
Open this post in threaded view
|

JSON Object to CSV Question

Joonas Liik
You say you are taking this from an xml file and want to get a CSV file..

Why are you making an intermediate JSON file?
Why do you need the CSV output?
Could you perhaps be better off using another format?

Your data seems to be a quite deeply nested hierarchical structure and
doesn't
  seem to suit the simple CSV format very well.. (i see several layers of
nested arrays for one)..
  how to you plan to map these nested structures to the CSV format?


And your example json bit.. the JSON you posted in your last post seems
quite different from some of the first ones you posted.

You cant expect to give people 5% of a malformed simplified example and to
get anything useful (much less usable) back.

And finally, the original xml? mock at least?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150619/8ba8b4b1/attachment.html>

Reply | Threaded
Open this post in threaded view
|

JSON Object to CSV Question

Saran Ahluwalia
My answers are below in red.

On Fri, Jun 19, 2015 at 2:47 PM, Joonas Liik <liik.joonas at gmail.com> wrote:

>
> You say you are taking this from an xml file and want to get a CSV file..
>
> Why are you making an intermediate JSON file?
>

I thought that this would be the most efficient method as it emulates
Python dictionaries. In addition the json module has numerous functional
purposes.


> Why do you need the CSV output?
>

Client requirements


Could you perhaps be better off using another format?
>

Perhaps...not sure if it worth revisiting at this point.

>
> Your data seems to be a quite deeply nested hierarchical structure and
> doesn't
>   seem to suit the simple CSV format very well.. (i see several layers of
> nested arrays for one)..
>   how to you plan to map these nested structures to the CSV format?
>
>
 That's my greatest challenge. From what I understand, I need to replicate
each key and associate it with n number of items in the array. I have been
able to address by denesting by one level using this function:



   1. def flatten(d, parent_key=''):
   2. items = []
   3. for k, v in d.items():
   4. try:
   5. items.extend(flatten(v, '%s%s_' % (parent_key, k)).items())
   6. except AttributeError:
   7. items.append(('%s%s' % (parent_key, k), v))
   8. return dict(items)
   9.
   10. final = (flatten(data2, parent_key =''))



For example:


   1. data2 = {
   2.
   3. "OTF": "0",
   4. "F": "False",
   5. "F": {
   6. "Int32": ["0",
   7. "0",
   8. "0",
   9. "0",
   10. "0",
   11. "0",
   12. "0",
   13. "0",
   14. "0",
   15. "0",
   16. "0",
   17. "0",
   18. "0",
   19. "0",
   20. "0",
   21. "0",
   22. "0",
   23. "0",
   24. "0",
   25. "0",
   26. "0",
   27. "0",
   28. "0",
   29. "0",
   30. "0"]
   31. },
   32. "D": {
   33. "B": ["0",
   34. "0",
   35. "0",
   36. "0",
   37. "0",
   38. "0",
   39. "0",
   40. "0",
   41. "0",
   42. "0",
   43. "0"]
   44. },
   45.
   46. "PBDS": {
   47. "DateTime": ["1/1/0001 12:00:00 AM",
   48. "1/1/0001 12:00:00 AM",
   49. "1/1/0001 12:00:00 AM",
   50. "1/1/0001 12:00:00 AM",
   51. "1/1/0001 12:00:00 AM",
   52. "1/1/0001 12:00:00 AM",
   53. "1/1/0001 12:00:00 AM",
   54. "1/1/0001 12:00:00 AM"]
   55. },
   56.
   57. "PBDS": {
   58. "Double": ["0",
   59. "0",
   60. "0",
   61. "0",
   62. "0",
   63. "0",
   64. "0",
   65. "0"]
   66. },
   67.
   68. "SCS": {
   69. "String": ["1",
   70. "2"]
   71. }
   72.
   73. }



is turned into


   1. {'D_B': ['0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0'],
   2. 'F_Int32': ['0',
   3. '0',
   4. '0',
   5. '0',
   6. '0',
   7. '0',
   8. '0',
   9. '0',
   10. '0',
   11. '0',
   12. '0',
   13. '0',
   14. '0',
   15. '0',
   16. '0',
   17. '0',
   18. '0',
   19. '0',
   20. '0',
   21. '0',
   22. '0',
   23. '0',
   24. '0',
   25. '0',
   26. '0'],
   27. 'OTF': '0',
   28. 'PBDS_Double': ['0', '0', '0', '0', '0', '0', '0', '0'],
   29. 'SCS_String': ['1', '2']}


I would need an output that maps each key with each item in the array.

 DB1 : 0, DB2: 0, DB3: 0 etc. and F1: 0, F1: 0. DB1, DB2 would be the
headers and the 0s as values in the CSV file.


> And your example json bit.. the JSON you posted in your last post seems
> quite different from some of the first ones you posted.
>
> You cant expect to give people 5% of a malformed simplified example and to
> get anything useful (much less usable) back.
>
> And finally, the original xml? mock at least?
>

All of our XML from our client is malformed.

>
>
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150619/53fe677a/attachment.html>

Reply | Threaded
Open this post in threaded view
|

JSON Object to CSV Question

Joonas Liik
this.. might not throw an eror, but you have 2 keys with the same name
"F", and 1 of them will probably be disgarded..., you have data
corruption even before you try to process it.

{
 "F": "False",
"F": {
"Int32": ["0",
"0",
"0"]
},
 }

you mentioned Excel at one point.
perhaps you could mock up what you'd like your finished data to look
like in a spreadsheet (google docs for instance, since thats easy to
link to) and reference there.

just having a list of headers doesnt say much about the data format you want.

"client wants csv" hmm..they want "csv" or they want "csv that fists
this very particular description that fits our special decoder or the
like" ?

do you know how the client will use this data. could that info be used
to simplify the output to some degree?

and finally..
the client gives you malformed xml??
I'm very sorry to hear that. also does the client know they are
emitting invalid xml?
    is it rly xml?
    is it valid in another language? (html is much more lenient for
instance, an html parser might be able to gleam more meaning)
    by what definition is it malformed? is it outright structuralyl
broken, does it fail to meet some schema?
                      does it fail to meet some expectation you have
for some reason ("client said it has these properties")

you also mentioned you use JSON because it maps nicely to python
dicts.. this is true ofc.. but why not just read that in to a python
dict in the first place?

> DB1 : 0, DB2: 0, DB3: 0 etc. and F1: 0, F1: 0. DB1, DB2 would be the headers and the 0s as values in the CSV file.

DB1 etc seems ok at first glance however...
say there are 2 nested items and each of them have a DB property which
is an array, you will have name collisions.
you need more thought in to naming the headers at the very least.

if this is meant for a spreadsheet.. then you will end up with 2 very
very very long rows, it will NOT be readable by any stretch of the
imagination.. do you want this.

i'm afraid you'll essentially end up with a translation that looks sth like

{A:{b:"c",q:"w"}}
===============
"A.b", "A.q"
"c", "w"

if you just want key-value pairs there are better options out
there..besides csv..

Reply | Threaded
Open this post in threaded view
|

JSON Object to CSV Question

Saran Ahluwalia
@Joonas:

The previous example was a typo. Please use the below example as a case
study.


   1. {'D_B': ['0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0'],
   2. 'F_Int32': ['0',
   3. '0',
   4. '0',
   5. '0',
   6. '0',
   7. '0',
   8. '0',
   9. '0',
   10. '0',
   11. '0',
   12. '0',
   13. '0',
   14. '0',
   15. '0',
   16. '0',
   17. '0',
   18. '0',
   19. '0',
   20. '0',
   21. '0',
   22. '0',
   23. '0',
   24. '0',
   25. '0',
   26. '0'],
   27. 'OTF': '0',
   28. 'PBDS_Double': ['0', '0', '0', '0', '0', '0', '0', '0'],
   29. 'SCS_String': ['1', '2']}
   30.


All of the questions regarding XML I have asked. I have to work within
their parameters. The CSV, for example may look like this:

DLA,FC,PC,WC,CN,Description,Code,CMC
0,00000,0,0,,,0,0

I have made some head way, as stated previously. I have now been able to
experiment with a list comprehension that will demonstrates how each header
will need  to be for headers with repeating values.


   1. >>> key = 'spam'
   2. >>> L = ['foo', 'bar', 'baz']
   3. >>> [('{}{}'.format(key, i), value) for i, value in enumerate(L, 1)]
   4. [('spam1', 'foo'), ('spam2', 'bar'), ('spam3', 'baz')]



Is there a way that you know of that will:

1) Allow me to preserve the following two functions:

port json
import sys

def hook(obj):
    return obj

def flatten(obj):
    for k, v in obj:
        if isinstance(v, list):
            yield from flatten(v)
        else:
            yield k, v

if __name__ == "__main__":
    with open("somefileneame.json") as f:
        data = json.load(f, object_pairs_hook=hook)

    pairs = list(flatten(data))

    writer = csv.writer(sys.stdout)
    header = writer.writerow([k for k, v in pairs])
    row = writer.writerow([v for k, v in pairs]) #writer.writerows for any
other iterable object




and

2)

Will conditionally allow me to recursively check if a key has a nested
array. If so, then apply the  [('{}{}'.format(key, i), value) for i, value
in enumerate(L, 1)] list comprehension?


On Fri, Jun 19, 2015 at 4:32 PM, Joonas Liik <liik.joonas at gmail.com> wrote:

> this.. might not throw an eror, but you have 2 keys with the same name
> "F", and 1 of them will probably be disgarded..., you have data
> corruption even before you try to process it.
>
> {
>  "F": "False",
> "F": {
> "Int32": ["0",
> "0",
> "0"]
> },
>  }
>
> you mentioned Excel at one point.
> perhaps you could mock up what you'd like your finished data to look
> like in a spreadsheet (google docs for instance, since thats easy to
> link to) and reference there.
>
> just having a list of headers doesnt say much about the data format you
> want.
>
> "client wants csv" hmm..they want "csv" or they want "csv that fists
> this very particular description that fits our special decoder or the
> like" ?
>
> do you know how the client will use this data. could that info be used
> to simplify the output to some degree?
>
> and finally..
> the client gives you malformed xml??
> I'm very sorry to hear that. also does the client know they are
> emitting invalid xml?
>     is it rly xml?
>     is it valid in another language? (html is much more lenient for
> instance, an html parser might be able to gleam more meaning)
>     by what definition is it malformed? is it outright structuralyl
> broken, does it fail to meet some schema?
>                       does it fail to meet some expectation you have
> for some reason ("client said it has these properties")
>
> you also mentioned you use JSON because it maps nicely to python
> dicts.. this is true ofc.. but why not just read that in to a python
> dict in the first place?
>
> > DB1 : 0, DB2: 0, DB3: 0 etc. and F1: 0, F1: 0. DB1, DB2 would be the
> headers and the 0s as values in the CSV file.
>
> DB1 etc seems ok at first glance however...
> say there are 2 nested items and each of them have a DB property which
> is an array, you will have name collisions.
> you need more thought in to naming the headers at the very least.
>
> if this is meant for a spreadsheet.. then you will end up with 2 very
> very very long rows, it will NOT be readable by any stretch of the
> imagination.. do you want this.
>
> i'm afraid you'll essentially end up with a translation that looks sth like
>
> {A:{b:"c",q:"w"}}
> ===============
> "A.b", "A.q"
> "c", "w"
>
> if you just want key-value pairs there are better options out
> there..besides csv..
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150619/5ecd620a/attachment.html>

Reply | Threaded
Open this post in threaded view
|

JSON Object to CSV Question

Ned Batchelder
In reply to this post by Joonas Liik
On Saturday, June 20, 2015 at 2:10:51 AM UTC-4, Sahlusar wrote:

> @Joonas:
>
>
> The previous example was a typo. Please use the below example as a case study.?
>
>
>
>
> {'D_B': ['0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0'], 'F_Int32': ['0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0'], 'OTF': '0', 'PBDS_Double': ['0', '0', '0', '0', '0', '0', '0', '0'], 'SCS_String': ['1', '2']}
>
>
> All of the questions regarding XML I have asked. I have to work within their parameters. The CSV, for example may look like this:
>
>
> DLA,FC,PC,WC,CN,Description,Code,CMC?
> 0,00000,0,0,,,0,0

There isn't a single word in common between the Python dictionary you've
shown and the CSV sample output you've shown.  There's no way anyone can
tell what mapping you intend here.

--Ned.

Reply | Threaded
Open this post in threaded view
|

JSON Object to CSV Question

Saran Ahluwalia
Here is the dictionary that corresponds with the CSV printed to the console:


   1. {
   2. "PAC": {
   3. "Account": [{
   4. "PC": "0",
   5. "CMC": "0",
   6. "WC": "0",
   7. "DLA": "0",
   8. "CN": null,
   9. "FC": {
   10. "Int32": ["0",
   11. "0",
   12. "0",
   13. "0",
   14. "0"]
   15. },
   16. "F": {
   17. "Description": null,
   18. "Code": "0"
   19. }
   20. }]
   21. }
   22. }


On Sat, Jun 20, 2015 at 11:04 AM, Ned Batchelder <ned at nedbatchelder.com>
wrote:

> On Saturday, June 20, 2015 at 2:10:51 AM UTC-4, Sahlusar wrote:
> > @Joonas:
> >
> >
> > The previous example was a typo. Please use the below example as a case
> study.
> >
> >
> >
> >
> > {'D_B': ['0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0'],
> 'F_Int32': ['0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',
> '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',
> '0',  '0',  '0'], 'OTF': '0', 'PBDS_Double': ['0', '0', '0', '0', '0', '0',
> '0', '0'], 'SCS_String': ['1', '2']}
> >
> >
> > All of the questions regarding XML I have asked. I have to work within
> their parameters. The CSV, for example may look like this:
> >
> >
> > DLA,FC,PC,WC,CN,Description,Code,CMC
> > 0,00000,0,0,,,0,0
>
> There isn't a single word in common between the Python dictionary you've
> shown and the CSV sample output you've shown.  There's no way anyone can
> tell what mapping you intend here.
>
> --Ned.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150620/790203f8/attachment.html>