Newbie : Best data archive strategy

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Newbie : Best data archive strategy

Mickael.barbo
Hi,

I'd like to archive some data.

I came with this idea :


class DataModel(models.Model):
xxx

class DataModelArchive(DataModel):
pass


The DataModelArchive database created has only "a pointer" to DataModel.

I saw a post indicating that I'll have to first create an AbstractDataModel and inherit for it for both DataModelArchive and DataModel.

Well, what is the "best way" to archive data using Django ?

Thx for your attention

Micka

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/ef59e00a-3354-44f5-9c99-5ca85885026e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

RE: Newbie : Best data archive strategy

Matthew Pava

I think what you’re really looking for is an audit log.  There are some Django packages available that do that automatically.

 

You might find this one quite helpful, called Django Reversion:

https://github.com/etianen/django-reversion

 

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of [hidden email]
Sent: Tuesday, June 12, 2018 12:22 PM
To: Django users
Subject: Newbie : Best data archive strategy

 

Hi,

I'd like to archive some data.

I came with this idea :


class DataModel(models.Model):

xxx


class DataModelArchive(DataModel):

pass

 


The DataModelArchive database created has only "a pointer" to DataModel.

I saw a post indicating that I'll have to first create an AbstractDataModel and inherit for it for both DataModelArchive and DataModel.

Well, what is the "best way" to archive data using Django ?

Thx for your attention

Micka

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/ef59e00a-3354-44f5-9c99-5ca85885026e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/7f06b4e05b3443ebbe20084a6c786a20%40ISS1.ISS.LOCAL.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Newbie : Best data archive strategy

Mickael.barbo
Thanks Matthew for your response.

I looked at Reversion. If I understand correctly what it did, it duplicate each transaction on internal database ?

Le mardi 12 juin 2018 20:01:02 UTC+2, Matthew Pava a écrit :

I think what you’re really looking for is an audit log.  There are some Django packages available that do that automatically.

 

You might find this one quite helpful, called Django Reversion:

<a href="https://github.com/etianen/django-reversion" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fetianen%2Fdjango-reversion\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEF1HeBUmDYUsPkh6ucElrmMVk_nA&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fetianen%2Fdjango-reversion\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEF1HeBUmDYUsPkh6ucElrmMVk_nA&#39;;return true;">https://github.com/etianen/django-reversion

 

 

From: <a href="javascript:" target="_blank" gdf-obfuscated-mailto="jTJ4TqcfBQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">django...@... [mailto:<a href="javascript:" target="_blank" gdf-obfuscated-mailto="jTJ4TqcfBQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">django...@googlegroups.com] On Behalf Of <a href="javascript:" target="_blank" gdf-obfuscated-mailto="jTJ4TqcfBQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">mickae...@...
Sent: Tuesday, June 12, 2018 12:22 PM
To: Django users
Subject: Newbie : Best data archive strategy

 

Hi,

I'd like to archive some data.

I came with this idea :


class DataModel(models.Model):

xxx


class DataModelArchive(DataModel):

pass

 


The DataModelArchive database created has only "a pointer" to DataModel.

I saw a post indicating that I'll have to first create an AbstractDataModel and inherit for it for both DataModelArchive and DataModel.

Well, what is the "best way" to archive data using Django ?

Thx for your attention

Micka

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="jTJ4TqcfBQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">django-users...@googlegroups.com.
To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="jTJ4TqcfBQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;"> djang...@....
Visit this group at <a href="https://groups.google.com/group/django-users" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/group/django-users&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/group/django-users&#39;;return true;">https://groups.google.com/group/django-users.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/django-users/ef59e00a-3354-44f5-9c99-5ca85885026e%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/django-users/ef59e00a-3354-44f5-9c99-5ca85885026e%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/django-users/ef59e00a-3354-44f5-9c99-5ca85885026e%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;"> https://groups.google.com/d/msgid/django-users/ef59e00a-3354-44f5-9c99-5ca85885026e%40googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/86e18583-0376-43f2-8419-b557459b5ca7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

RE: Newbie : Best data archive strategy

Matthew Pava

Hi Mickael,

It saves a copy of each record every time a record is changed.  But it is saved in a separate table, so it doesn’t change your model’s table.

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of [hidden email]
Sent: Tuesday, June 12, 2018 1:17 PM
To: Django users
Subject: Re: Newbie : Best data archive strategy

 

Thanks Matthew for your response.

I looked at Reversion. If I understand correctly what it did, it duplicate each transaction on internal database ?

Le mardi 12 juin 2018 20:01:02 UTC+2, Matthew Pava a écrit :

I think what you’re really looking for is an audit log.  There are some Django packages available that do that automatically.

 

You might find this one quite helpful, called Django Reversion:

https://github.com/etianen/django-reversion

 

 

From: <a href="javascript:" target="_blank">django...@... [mailto:<a href="javascript:" target="_blank">django...@...] On Behalf Of <a href="javascript:" target="_blank">mickae...@...
Sent: Tuesday, June 12, 2018 12:22 PM
To: Django users
Subject: Newbie : Best data archive strategy

 

Hi,

I'd like to archive some data.

I came with this idea :


class DataModel(models.Model):

xxx


class DataModelArchive(DataModel):

pass

 


The DataModelArchive database created has only "a pointer" to DataModel.

I saw a post indicating that I'll have to first create an AbstractDataModel and inherit for it for both DataModelArchive and DataModel.

Well, what is the "best way" to archive data using Django ?

Thx for your attention

Micka

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank">django-users...@....
To post to this group, send email to <a href="javascript:" target="_blank">djang...@....
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/ef59e00a-3354-44f5-9c99-5ca85885026e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/86e18583-0376-43f2-8419-b557459b5ca7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/148f20f481e5409a833ea043f771cbda%40ISS1.ISS.LOCAL.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Newbie : Best data archive strategy

Gerardo Palazuelos Guerrero
Hi Mickael,
When I read Archive, in my mind is a different thing, not the same as audit. So, for one moment, think about situation/scenario you are working on.

Archive is to manage data purge from my transactional tables; this means I want to "delete" data (that has not more value) or I want to "move" data to an archive system (data is still relevant and useful).

Audit is to create a perfect tracking of transitions on value overtime. This is required due to special regulations you have.

So, it might be useful if you clarify what is the scenario you are trying to solve.

Regards,
Gerardo.



--
Gerardo Palazuelos Guerrero


On Tue, Jun 12, 2018 at 12:14 PM, Matthew Pava <[hidden email]> wrote:

Hi Mickael,

It saves a copy of each record every time a record is changed.  But it is saved in a separate table, so it doesn’t change your model’s table.

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of [hidden email]
Sent: Tuesday, June 12, 2018 1:17 PM
To: Django users
Subject: Re: Newbie : Best data archive strategy

 

Thanks Matthew for your response.

I looked at Reversion. If I understand correctly what it did, it duplicate each transaction on internal database ?

Le mardi 12 juin 2018 20:01:02 UTC+2, Matthew Pava a écrit :

I think what you’re really looking for is an audit log.  There are some Django packages available that do that automatically.

 

You might find this one quite helpful, called Django Reversion:

https://github.com/etianen/django-reversion

 

 

From: [hidden email] [mailto:django...@googlegroups.com] On Behalf Of [hidden email]
Sent: Tuesday, June 12, 2018 12:22 PM
To: Django users
Subject: Newbie : Best data archive strategy

 

Hi,

I'd like to archive some data.

I came with this idea :


class DataModel(models.Model):

xxx


class DataModelArchive(DataModel):

pass

 


The DataModelArchive database created has only "a pointer" to DataModel.

I saw a post indicating that I'll have to first create an AbstractDataModel and inherit for it for both DataModelArchive and DataModel.

Well, what is the "best way" to archive data using Django ?

Thx for your attention

Micka

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/ef59e00a-3354-44f5-9c99-5ca85885026e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/86e18583-0376-43f2-8419-b557459b5ca7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/148f20f481e5409a833ea043f771cbda%40ISS1.ISS.LOCAL.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAJ8iCyNj%2B9Q8%2BNJaE5hP9CFgFJhyjFgh-SMZZ1RPm9tYO%2BaczA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Newbie : Best data archive strategy

Mickael.barbo
Ok, so "audit log" is like a "snapshot". I think it's too much for what I need.

To clarify my needs, I have a data that once "used/read", it's no more needed.
I just want to keep it "archived", in an other place to get access to it if I want to add some function (statistics, etc...).

So, Gerardo, it seems to be like " I want to "move" data to an archive system (data is still relevant and useful)."

Hope this clarify my needs...

Thx Matthew and Gerardo for your help.


2018-06-12 20:38 GMT+02:00 Gerardo Palazuelos Guerrero <[hidden email]>:
Hi Mickael,
When I read Archive, in my mind is a different thing, not the same as audit. So, for one moment, think about situation/scenario you are working on.

Archive is to manage data purge from my transactional tables; this means I want to "delete" data (that has not more value) or I want to "move" data to an archive system (data is still relevant and useful).

Audit is to create a perfect tracking of transitions on value overtime. This is required due to special regulations you have.

So, it might be useful if you clarify what is the scenario you are trying to solve.

Regards,
Gerardo.



--
Gerardo Palazuelos Guerrero


On Tue, Jun 12, 2018 at 12:14 PM, Matthew Pava <[hidden email]> wrote:

Hi Mickael,

It saves a copy of each record every time a record is changed.  But it is saved in a separate table, so it doesn’t change your model’s table.

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of [hidden email]
Sent: Tuesday, June 12, 2018 1:17 PM
To: Django users
Subject: Re: Newbie : Best data archive strategy

 

Thanks Matthew for your response.

I looked at Reversion. If I understand correctly what it did, it duplicate each transaction on internal database ?

Le mardi 12 juin 2018 20:01:02 UTC+2, Matthew Pava a écrit :

I think what you’re really looking for is an audit log.  There are some Django packages available that do that automatically.

 

You might find this one quite helpful, called Django Reversion:

https://github.com/etianen/django-reversion

 

 

From: [hidden email] [mailto:django...@googlegroups.com] On Behalf Of [hidden email]
Sent: Tuesday, June 12, 2018 12:22 PM
To: Django users
Subject: Newbie : Best data archive strategy

 

Hi,

I'd like to archive some data.

I came with this idea :


class DataModel(models.Model):

xxx


class DataModelArchive(DataModel):

pass

 


The DataModelArchive database created has only "a pointer" to DataModel.

I saw a post indicating that I'll have to first create an AbstractDataModel and inherit for it for both DataModelArchive and DataModel.

Well, what is the "best way" to archive data using Django ?

Thx for your attention

Micka

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]om.
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/ef59e00a-3354-44f5-9c99-5ca85885026e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/86e18583-0376-43f2-8419-b557459b5ca7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/148f20f481e5409a833ea043f771cbda%40ISS1.ISS.LOCAL.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Django users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-users/bV_Wf8qanvQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAJ8iCyNj%2B9Q8%2BNJaE5hP9CFgFJhyjFgh-SMZZ1RPm9tYO%2BaczA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAPLHwtzdYRFsrzTyzsNE8h34xm66YjZjOtR-ZTW_Q8N6iUiA_Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.