Learning notes for the migrations feature in the Django framework (3)

In the previous two notes, I learned a lot about django's migrations feature. This time, let's learn about troubleshooting migrations that is useful for operations.

Add unique field to state with legacy data

Problem Description: The data was created after running the site for a period of time. Now add a new unique, non-nullable field to your model. In order for the model to be reflected in the database, you need to give default values to the legacy data (existing data). However, giving multiple data records the same default value violates the uniqueness rule and causes an error. If you're in a development environment, you can delete all the databases and migration files and recreate the database, but let's solve this problem in a more graceful way.

Problem reproduction

Go into the Django shell and create the demo data.

python manage.py shell
>>> from polls.models import Category, Article
>>> c = Category(name="game")
>>> c.save()
>>> a1 = Article(title="play Game1", text="Game1 is amazing!", category=c, image_url = "http://url1")
>>> a1.save()
>>> a2 = Article(title="play Game2", text="Game2 is good!", category=c, image_url = "http://url2")
>>> a2.save()
>>> a3 = Article(title="play Game3", text="Game3 is dump!", category=c, image_url = "http://url3")
>>> a3.save()

You can see the data created by the mysql client (first specify the project database). First of all, blog category data:

mysql> select * from polls_category;
+----+------+
| id | name |
+----+------+
|  1 | game |
+----+------+
1 row in set (0.00 sec)

Next, sentence data

mysql> select * from polls_article;
+----+------------+-------------------+-------------+-------------+
| id | title      | text              | category_id | image_url   |
+----+------------+-------------------+-------------+-------------+
|  1 | play Game1 | Game1 is amazing! |           1 | http://url1 |
|  2 | play Game2 | Game2 is good!    |           1 | http://url2 |
|  3 | play Game3 | Game3 is dump!    |           1 | http://url3 |
+----+------------+-------------------+-------------+-------------+
3 rows in set (0.00 sec)

Now add an article_id field to the Article model to uniquely identify the Article (unlike the auto-created primary key). This field meets the following requirements: --non-nullable (Null is not allowed because it is an identifier, default value is required) --unique (makes it uniquely identifiable) If you change the model file according to your requirements:

models.py

class Article(models.Model):
    title = models.CharField(max_length=100)
    text = models.CharField(max_length=1000)
    category = models.ForeignKey(Category, on_delete=models.CASCADE)
    image_url = models.URLField(max_length=200, default='toBeImplement')
    article_id = models.UUIDField(default=uuid.uuid4(), unique=True)

(Only Article class is displayed) Create migrations file:

python manage.py makemigrations polls

success

Migrations for 'polls':
  polls/migrations/0003_article_article_id.py
    - Add field article_id to article

Trying to reflect in the database:

python manage.py migrate polls

I get an error (partial log)

...
django.db.utils.IntegrityError: (1062, "Duplicate entry '3059fd8259254f0693ce8d91cf1198cc' for key 'article_id'")

To figure out the error, take a look at the migrations file 0003_article_article_id.py:

0003_article_article_id.py

# -*- coding: utf-8 -*-
# Generated by Django 1.11.2 on 2017-07-18 08:25
from __future__ import unicode_literals

from django.db import migrations, models
import uuid


class Migration(migrations.Migration):

    dependencies = [
        ('polls', '0002_article_image_url'),
    ]

    operations = [
        migrations.AddField(
            model_name='article',
            name='article_id',
            field=models.UUIDField(default=uuid.UUID('3059fd82-5925-4f06-93ce-8d91cf1198cc'), unique=True),
        ),
    ]

The reason for the error is that when I added article_id to an existing data (Article) record, I gave it the same default value and violated the unique requirement.

How to fix

If you have legacy data, you need three steps to add a non-nullable and unique field. Refer to the official site

Overview

Assumption: Added the corresponding field to the model file.

1. Create a migrations file that corresponds to the model change and change the field attributes from unique to nullable.
2. Create a blank migratioins file and give the legacy data a default value.
3. Create a blank migrations file and set the field attributes back to unique.

Details

Step1: As mentioned above, the fields have already been added to the Article model and the migrations file 0003_article_article_id.py has also been created. Therefore, modify the migrations file 0003_article_article_id.py.

0003_article_article_id.py

# -*- coding: utf-8 -*-
# Generated by Django 1.11.2 on 2017-07-18 08:25
from __future__ import unicode_literals

from django.db import migrations, models
import uuid


class Migration(migrations.Migration):

    dependencies = [
        ('polls', '0002_article_image_url'),
    ]

    operations = [
        migrations.AddField(
            model_name='article',
            name='article_id',
            field=models.UUIDField(default=uuid.UUID('3059fd82-5925-4f06-93ce-8d91cf1198cc'), null=True),
        ),
    ]

That is, change "unique = True" in class Migration-> operations-> migrations.AddField () to "null = True". You can now put the same default values in the new column of legacy data.

Step2: Create a blank migrations file.

python manage.py makemigrations polls --empty

Now put a unique value in each record of the legacy data in a new column

0004_auto_20170718_0901.py

# -*- coding: utf-8 -*-
# Generated by Django 1.11.2 on 2017-07-18 09:01
from __future__ import unicode_literals

from django.db import migrations
import uuid


def gen_uuid(apps, schema_editor):
    Article = apps.get_model('polls', 'Article')
    for row in Article.objects.all():
        row.article_id = uuid.uuid4()
        row.save(update_fields=['article_id'])


class Migration(migrations.Migration):

    dependencies = [
        ('polls', '0003_article_article_id'),
    ]

    operations = [
        migrations.RunPython(gen_uuid, reverse_code=migrations.RunPython.noop)
    ]

Step3: Finally, create a blank migrations file again and set the column attributes back to unique. Now you can limit the columns of the newly created data record.

0005_auto_20170718_0901.py

# -*- coding: utf-8 -*-
# Generated by Django 1.11.2 on 2017-07-18 09:01
from __future__ import unicode_literals

from django.db import migrations, models
import uuid


class Migration(migrations.Migration):

    dependencies = [
        ('polls', '0004_auto_20170718_0901'),
    ]

    operations = [
        migrations.AlterField(
            model_name='article',
            name='article_id',
            field=models.UUIDField(default=uuid.uuid4, unique=True),
        ),
    ]

Once the migrations file is ready, reflect the changes to the model and legacy data.

(a_site) IT-02085-M:m_Migrate ruofan.ye$ python manage.py migrate polls
Operations to perform:
  Apply all migrations: polls
Running migrations:
  Applying polls.0003_article_article_id... OK
  Applying polls.0004_auto_20170718_0901... OK
  Applying polls.0005_auto_20170718_0901... OK

Success, and looking at the database

mysql> select * from polls_article;
+----+------------+-------------------+-------------+-------------+----------------------------------+
| id | title      | text              | category_id | image_url   | article_id                       |
+----+------------+-------------------+-------------+-------------+----------------------------------+
|  1 | play Game1 | Game1 is amazing! |           1 | http://url1 | 7347eab9e4df47eb989de30fc6baeec9 |
|  2 | play Game2 | Game2 is good!    |           1 | http://url2 | c4bac1cc1d3242aa934c9cb16f1cf5a9 |
|  3 | play Game3 | Game3 is dump!    |           1 | http://url3 | e6094b7b24fc46e7a59a7b90aaca298c |
+----+------------+-------------------+-------------+-------------+----------------------------------+
3 rows in set (0.00 sec)

I was able to successfully add a new column to the table and legacy data. Create new Article data:

>>> a4 = Article(title="play Game4", text="Game4 is new-feeling!", category=c, image_url = "http://url4")
>>> a4.save()

Looking at the database:

mysql> select * from polls_article;
+----+------------+-----------------------+-------------+-------------+----------------------------------+
| id | title      | text                  | category_id | image_url   | article_id                       |
+----+------------+-----------------------+-------------+-------------+----------------------------------+
|  1 | play Game1 | Game1 is amazing!     |           1 | http://url1 | 7347eab9e4df47eb989de30fc6baeec9 |
|  2 | play Game2 | Game2 is good!        |           1 | http://url2 | c4bac1cc1d3242aa934c9cb16f1cf5a9 |
|  3 | play Game3 | Game3 is dump!        |           1 | http://url3 | e6094b7b24fc46e7a59a7b90aaca298c |
|  4 | play Game4 | Game4 is new-feeling! |           1 | http://url4 | 0f1fcf070c544784b2a11ba2a4661cd7 |
+----+------------+-----------------------+-------------+-------------+----------------------------------+
4 rows in set (0.00 sec)

The article_id field was implemented as expected.