Django app taking too long to upload data into postgresql database

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Django app taking too long to upload data into postgresql database

Guillermo Yáñez Feliú

Hello,


I’m working in a project that consists in converting a local postgresql database (that uses sqlalchemy as the ORM) into a web application, in which I upload excel sheets, read them, do some small cleaning and then upload selected data into a postgresql database using Django’s ORM. The idea is to have the data in a server instead of in every user’s machine.

Everything is ok but data loading is taking too long since, I think, I am using panda’s dataframes to easily structure, read and save the data. In the local version of the library, I used lists and was way faster.


I don’t if it’s related to Sqlalchemy, Django, lists or dataframes. Any suggestions on how to read spreadsheets data and upload it into a postgresql database using Django?


Thanks a lot.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/b8339488-0ba5-4f76-9212-be745c07ba54%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Django app taking too long to upload data into postgresql database

Adam Johnson-2
This mailing list is for the development of Django itself, not for support using Django. Please use the django-users mailing list for that, or IRC #django on freenode, or a site like Stack Overflow.

On Fri, 15 Mar 2019 at 18:25, Guillermo Yáñez Feliú <[hidden email]> wrote:

Hello,


I’m working in a project that consists in converting a local postgresql database (that uses sqlalchemy as the ORM) into a web application, in which I upload excel sheets, read them, do some small cleaning and then upload selected data into a postgresql database using Django’s ORM. The idea is to have the data in a server instead of in every user’s machine.

Everything is ok but data loading is taking too long since, I think, I am using panda’s dataframes to easily structure, read and save the data. In the local version of the library, I used lists and was way faster.


I don’t if it’s related to Sqlalchemy, Django, lists or dataframes. Any suggestions on how to read spreadsheets data and upload it into a postgresql database using Django?


Thanks a lot.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/b8339488-0ba5-4f76-9212-be745c07ba54%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


--
Adam

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyDDM3yuM1iVnO%2BgTt6tV1-%3DS%2BmTYp2tvqkarjFzYTEvFCwbg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.