Learning notes for the migrations feature in the Django framework (3)

In the previous two notes, I learned a lot about django's migrations feature. This time, let's learn about troubleshooting migrations that is useful for operations.

Development environment
Mac OS:Sierra
python2.7.10
django1.11.2
mysql5.7.18

Add unique field to state with legacy data

Problem Description: The data was created after running the site for a period of time. Now add a new unique, non-nullable field to your model. In order for the model to be reflected in the database, you need to give default values to the legacy data (existing data). However, giving multiple data records the same default value violates the uniqueness rule and causes an error. If you're in a development environment, you can delete all the databases and migration files and recreate the database, but let's solve this problem in a more graceful way.

Problem reproduction

Go into the Django shell and create the demo data.

python manage.py shell
>>> from polls.models import Category, Article
>>> c = Category(name="game")
>>> c.save()
>>> a1 = Article(title="play Game1", text="Game1 is amazing!", category=c, image_url = "http://url1")
>>> a1.save()
>>> a2 = Article(title="play Game2", text="Game2 is good!", category=c, image_url = "http://url2")
>>> a2.save()
>>> a3 = Article(title="play Game3", text="Game3 is dump!", category=c, image_url = "http://url3")
>>> a3.save()

You can see the data created by the mysql client (first specify the project database). First of all, blog category data:

mysql> select * from polls_category;
+----+------+
| id | name |
+----+------+
|  1 | game |
+----+------+
1 row in set (0.00 sec)

Next, sentence data

mysql> select * from polls_article;
+----+------------+-------------------+-------------+-------------+
| id | title      | text              | category_id | image_url   |
+----+------------+-------------------+-------------+-------------+
|  1 | play Game1 | Game1 is amazing! |           1 | http://url1 |
|  2 | play Game2 | Game2 is good!    |           1 | http://url2 |
|  3 | play Game3 | Game3 is dump!    |           1 | http://url3 |
+----+------------+-------------------+-------------+-------------+
3 rows in set (0.00 sec)

Now add an article_id field to the Article model to uniquely identify the Article (unlike the auto-created primary key). This field meets the following requirements: --non-nullable (Null is not allowed because it is an identifier, default value is required) --unique (makes it uniquely identifiable) If you change the model file according to your requirements:

models.py


class Article(models.Model):
    title = models.CharField(max_length=100)
    text = models.CharField(max_length=1000)
    category = models.ForeignKey(Category, on_delete=models.CASCADE)
    image_url = models.URLField(max_length=200, default='toBeImplement')
    article_id = models.UUIDField(default=uuid.uuid4(), unique=True)

(Only Article class is displayed) Create migrations file:

python manage.py makemigrations polls

success

Migrations for 'polls':
  polls/migrations/0003_article_article_id.py
    - Add field article_id to article

Trying to reflect in the database:

python manage.py migrate polls

I get an error (partial log)

...
django.db.utils.IntegrityError: (1062, "Duplicate entry '3059fd8259254f0693ce8d91cf1198cc' for key 'article_id'")

To figure out the error, take a look at the migrations file 0003_article_article_id.py:

0003_article_article_id.py


# -*- coding: utf-8 -*-
# Generated by Django 1.11.2 on 2017-07-18 08:25
from __future__ import unicode_literals

from django.db import migrations, models
import uuid


class Migration(migrations.Migration):

    dependencies = [
        ('polls', '0002_article_image_url'),
    ]

    operations = [
        migrations.AddField(
            model_name='article',
            name='article_id',
            field=models.UUIDField(default=uuid.UUID('3059fd82-5925-4f06-93ce-8d91cf1198cc'), unique=True),
        ),
    ]

The reason for the error is that when I added article_id to an existing data (Article) record, I gave it the same default value and violated the unique requirement.

How to fix

If you have legacy data, you need three steps to add a non-nullable and unique field. Refer to the official site

Overview

Assumption: Added the corresponding field to the model file.

  1. Create a migrations file that corresponds to the model change and change the field attributes from unique to nullable.
  2. Create a blank migratioins file and give the legacy data a default value.
  3. Create a blank migrations file and set the field attributes back to unique.

Details

Step1: As mentioned above, the fields have already been added to the Article model and the migrations file 0003_article_article_id.py has also been created. Therefore, modify the migrations file 0003_article_article_id.py.

0003_article_article_id.py


# -*- coding: utf-8 -*-
# Generated by Django 1.11.2 on 2017-07-18 08:25
from __future__ import unicode_literals

from django.db import migrations, models
import uuid


class Migration(migrations.Migration):

    dependencies = [
        ('polls', '0002_article_image_url'),
    ]

    operations = [
        migrations.AddField(
            model_name='article',
            name='article_id',
            field=models.UUIDField(default=uuid.UUID('3059fd82-5925-4f06-93ce-8d91cf1198cc'), null=True),
        ),
    ]

That is, change "unique = True" in class Migration-> operations-> migrations.AddField () to "null = True". You can now put the same default values in the new column of legacy data.

Step2: Create a blank migrations file.

python manage.py makemigrations polls --empty

Now put a unique value in each record of the legacy data in a new column

0004_auto_20170718_0901.py


# -*- coding: utf-8 -*-
# Generated by Django 1.11.2 on 2017-07-18 09:01
from __future__ import unicode_literals

from django.db import migrations
import uuid


def gen_uuid(apps, schema_editor):
    Article = apps.get_model('polls', 'Article')
    for row in Article.objects.all():
        row.article_id = uuid.uuid4()
        row.save(update_fields=['article_id'])


class Migration(migrations.Migration):

    dependencies = [
        ('polls', '0003_article_article_id'),
    ]

    operations = [
        migrations.RunPython(gen_uuid, reverse_code=migrations.RunPython.noop)
    ]

Step3: Finally, create a blank migrations file again and set the column attributes back to unique. Now you can limit the columns of the newly created data record.

0005_auto_20170718_0901.py


# -*- coding: utf-8 -*-
# Generated by Django 1.11.2 on 2017-07-18 09:01
from __future__ import unicode_literals

from django.db import migrations, models
import uuid


class Migration(migrations.Migration):

    dependencies = [
        ('polls', '0004_auto_20170718_0901'),
    ]

    operations = [
        migrations.AlterField(
            model_name='article',
            name='article_id',
            field=models.UUIDField(default=uuid.uuid4, unique=True),
        ),
    ]

Once the migrations file is ready, reflect the changes to the model and legacy data.

(a_site) IT-02085-M:m_Migrate ruofan.ye$ python manage.py migrate polls
Operations to perform:
  Apply all migrations: polls
Running migrations:
  Applying polls.0003_article_article_id... OK
  Applying polls.0004_auto_20170718_0901... OK
  Applying polls.0005_auto_20170718_0901... OK

Success, and looking at the database

mysql> select * from polls_article;
+----+------------+-------------------+-------------+-------------+----------------------------------+
| id | title      | text              | category_id | image_url   | article_id                       |
+----+------------+-------------------+-------------+-------------+----------------------------------+
|  1 | play Game1 | Game1 is amazing! |           1 | http://url1 | 7347eab9e4df47eb989de30fc6baeec9 |
|  2 | play Game2 | Game2 is good!    |           1 | http://url2 | c4bac1cc1d3242aa934c9cb16f1cf5a9 |
|  3 | play Game3 | Game3 is dump!    |           1 | http://url3 | e6094b7b24fc46e7a59a7b90aaca298c |
+----+------------+-------------------+-------------+-------------+----------------------------------+
3 rows in set (0.00 sec)

I was able to successfully add a new column to the table and legacy data. Create new Article data:

>>> a4 = Article(title="play Game4", text="Game4 is new-feeling!", category=c, image_url = "http://url4")
>>> a4.save()

Looking at the database:

mysql> select * from polls_article;
+----+------------+-----------------------+-------------+-------------+----------------------------------+
| id | title      | text                  | category_id | image_url   | article_id                       |
+----+------------+-----------------------+-------------+-------------+----------------------------------+
|  1 | play Game1 | Game1 is amazing!     |           1 | http://url1 | 7347eab9e4df47eb989de30fc6baeec9 |
|  2 | play Game2 | Game2 is good!        |           1 | http://url2 | c4bac1cc1d3242aa934c9cb16f1cf5a9 |
|  3 | play Game3 | Game3 is dump!        |           1 | http://url3 | e6094b7b24fc46e7a59a7b90aaca298c |
|  4 | play Game4 | Game4 is new-feeling! |           1 | http://url4 | 0f1fcf070c544784b2a11ba2a4661cd7 |
+----+------------+-----------------------+-------------+-------------+----------------------------------+
4 rows in set (0.00 sec)

The article_id field was implemented as expected.

Recommended Posts

Learning notes for the migrations feature in the Django framework (2)
Learning notes for the migrations feature in the Django framework (3)
Learning notes for the migrations feature in the Django framework (1)
Miscellaneous notes about the Django REST framework
List method for nested resources in Django REST framework
Summary of stumbling blocks in Django for the first time
Launch notes for existing Django applications
Learning history for participating in team app development in Python ~ Django Tutorial 5 ~
Learning history for participating in team app development in Python ~ Django Tutorial 4 ~
Loop the For statement in reverse in an HTML file on Django
Learning history for participating in team app development in Python ~ Django Tutorial 1, 2, 3 ~
Switch the language displayed in Django 1.9
Learning history for participating in team app development in Python ~ Django Tutorial 6 ~
Learning history for participating in team app development in Python ~ Django Tutorial 7 ~
The meaning of ".object" in Django
Start Django for the first time
Summary of pages useful for studying the deep learning framework Chainer
Get the query string (query string) in Django
Get the client's IP address in Django
Introducing the BOT framework Minette for Python
Logical deletion in Django, DRF (Django REST Framework)
Understand the benefits of the Django Rest Framework
Django ~ Let's display it in the browser ~
Notes on nfc.ContactlessFrontend () for nfcpy in python
Learning notes from the beginning of Python 1
Notes on creating static files in Django
Change the list in a for statement
Same-Site attribute setting for cookies in Django
Get query parameters for GET requests in Django
MongoDB for the first time in Python
Notes for using python (pydev) in eclipse
Learning notes from the beginning of Python 2
Try hitting the Spotify API in Django.
[Reinforcement learning] Search for the best route
Redo everything for the Django login screen
django notes
I tried to predict the change in snowfall for 2 years by machine learning
Feature engineering for machine learning starting with the 4th Google Colaboratory --Interaction features
Django notes
How to install the deep learning framework Tensorflow 1.0 in the Anaconda environment of Windows
The _authenticate_with_backend function was obsolete in django auth.autenticate
CERTIFICATE_VERIFY_FAILED in Python 3.6, the official installer for macOS
The story of low learning costs for Python
Upgrade the Azure Machine Learning SDK for Python
Tips for hitting the ATND API in Python
Notes for implementing simple collaborative filtering in Python
Specify the view URL in your Django template
The story of viewing media files in Django
Implement JWT login functionality in Django REST framework
[Implementation for learning] Implement Stratified Sampling in Python (1)
[Django] css in the project cannot be read
Do an ambiguous search for mysql in Django
Knowledge notes needed to understand the Python framework
Implementing authentication in Django REST Framework with djoser
[For beginners] Introduction to vectorization in machine learning
I installed Chainer, a framework for deep learning
Let's reproduce the math teaching tool "Jamaica" ❗️ vol.02 "Notes for creating functions in Python"
Let's reproduce the math teaching tool "Jamaica" ❗️ vol.01 "Notes for displaying images in Python"