Migration to prod

[Jacky56]

It has been a long journey with piccolo ORM (still better than django).

I am currently performing a migration to a prod environment with +100 files. And I have notice that running piccolo migrations forward all do not run in timestamp creation order!

When you have a large project, you are bound to face dependencies of some columns not existing, or tables not created yet.

one of my microservices:

👍 17 migrations already complete
⏩ 17 migrations not yet run
🚀 Running 17 migrations:
  - 2024-06-09T00:03:28:809161 [forwards]... ok! ✔️
  - 2024-06-09T15:16:04:095641 [forwards]... ok! ✔️
  - 2024-06-09T20:52:44:531139 [forwards]... ok! ✔️
  - 2024-06-10T00:01:19:783644 [forwards]... ok! ✔️
  - 2024-06-12T23:36:50:424117 [forwards]... ok! ✔️
  - 2024-06-13T20:55:43:359308 [forwards]... ok! ✔️
  - 2024-06-14T22:40:25:381688 [forwards]... ok! ✔️
  - 2024-06-14T23:03:56:685730 [forwards]... ok! ✔️
  - 2024-06-17T21:17:29:684707 [forwards]... ok! ✔️
  - 2024-05-04T06:30:14:681376 [forwards]... inserting jobs
The command failed.
column "job_type" does not exist

I suspect that piccolo migrations forward all evaluates one module at a time instead of merging all modules and sorting by order

[dantownsend]

Do you have several apps?

The way it currently works is there are dependencies between apps e.g. run app X before app Y, but not really fine grained dependencies between migrations within different apps.

We could solve that by having more explicit dependencies, or maybe having a squash command or something to just flatten the migrations when you get loads of them.

[Jacky56]

hello @dantownsend

I haven't counted, but I assume +10 apps, I intend to split these apps into their own microservices (with their own controllers & dbs) in the future if needed.

We could solve that by having more explicit dependencies, or maybe having a squash command or something to just flatten the migrations when you get loads of them.

the latter option sounds appealing.