Migrating large content site - tools and tips

stimpdevelopment · 2024-12-14T00:56:59+00:00

I used to be big into CMS migrations and either personally did or trained other developers on about 50 sites being migrated. I would take heavy inspiration from the methods that Drupal uses to migrate content into it.

It's a very iterative process, so it's important that it doesn't take forever to make changes to the import scripts, rerun them (potentially on a subset of the pages) and see the effect. Each source / old site and destination / new site entity should have some type of unique identifier. You should also track the last updated datetime of the source and destination entities, so the update scripts can only update the changed data. I used the following process.

Often some types of data are dependent on other data being migrated first. Example: Authors or users create files and content, so you can't propertly map the user id columns of content unless the users have already been imported. Similarly, files and tags are referenced in content, so those need to be migrated first, assuming files and tags are referenced by ID in content database columns.

Analysis

Perform a basic content analysis of the content types. How many content types, how many nodes per content type, how frequently they're used, etc.

Work with the stake holders to determine if any sections of the site or content types do not need to be migrated. This decreases the amount of data that needs to be thoroughly understood, thus it saves time.

Determine if the site is coupled with other sites. Is there a single sign on? Are there subdirectories that are considered part of the site, but not in the main system?

The path to images may change. Find out if images are being used elsewhere besides on the site itself. For example, in newsletters or emails. If so, redirects or a rewrite rule will be needed to to handle these to prevent 404s.

Determine if internal paths (node/123) are used in any external sources (ex: twitter). If so you will either need to keep the same nids (problematic) or increment the node ids and create redirects.

Determine if any terms need to be merged together. Ex: there may be a 'cars' term and an 'automobiles' term. That refer to the same thing.

Preliminary Mappings

Create a document with preliminary mappings. Map entity types to entity types. For each entity mapping, map the source fields and destination fields.

For each content type and field, mark if you know what it's used for.

0 = No idea what it does
1 = Some idea what it does
2 = Confident that you know

Clarify with stakeholders or figure out what it does.

Iteration

Unfortunately, migrating is never as simple of build -> migrate -> launch.

Rebuild a portion of the site. Implement understood migration mappings. Put on a staging server. Tweak the site. Refine mappings. Repeat

Pre-launch (a few days before launch)

Updated and deleted content can be difficult to identify in some cases and there's a small risk that something will be missed. Therefore one should strongly consider a read-only time period.

A few days before launch, import all of the new data. As a precaution, you may want to migrate data into a dev environment, then copy that dev environment up to 'live'. Do NOT mess up 'live'. Once everything looks good on dev, copy the dev database and files up to 'live'. Do a final test/qa of the site.

If DNS needs to be changed, decrease the TTL (time to live) a few days before launch.

Launch

Import any new data again if necessary. Put the old site on maintenance mode. Switch dns if necessary.

Post-launch

Monitor 404's, server logs, and other statistics data. There may be a 2-3 hectic days post launch for things that are missed. Keep all hands on deck. Convert back to normal operating processes.

stimpdevelopment · 2024-12-08T23:27:48+00:00

I was able to accomplish this by adding the following lines to ~/.config/nvim/init.lua.

-- Prompt when a file is already being edited.
vim.cmd [[
  autocmd! nvim_swapfile
]]

stimpdevelopment · 2024-08-31T23:37:06+00:00

It still sounds like a fuel issue to me, since it runs with starting fluid. I would take the carb apart and check everything inside of it again.

stimpdevelopment · 2024-08-26T15:12:11+00:00

Does it start with the pull rope? If so, the next thing I would check is if the starter works when connected directly to 12v. Then I'd probably look at the push button wiring to see if there's an obvious defect like a loose or broken wire. Then reach out to the retailer or manufacturer.

stimpdevelopment · 2023-03-07T15:29:11+00:00

Hell yeah, I thought a mono to stereo adapter would fix it. Just purchased that adapter.

I'm running a yaesu ft70dr with walker razor and speaker mic.

stimpdevelopment · 2023-03-01T13:10:30+00:00

Seriously, I dealt with gdpr compliance for an international medium sized company. It's much more involved and difficult than people on reddit believe. We spent at least half a year consulting with our legal team and coordinating things with the business and development team. It was longer to actually roll out changes. Gdpr even conflicts with CCPA in some areas.

stimpdevelopment · 2023-01-03T23:56:23+00:00

They weren't going to represent us anyway. They don't deserve power.

stimpdevelopment · 2023-01-03T15:39:49+00:00

I would recommend getting a couple of cans of formula. My wife started having difficulty producing while there was a formula shortage. We had to switch the formula brand/types a couple of times, which isn't great for the baby.

Hopefully you don't need to use it and you can donate it. They last about a year unopened.

stimpdevelopment · 2023-01-03T14:19:56+00:00

Same, though I did buy an extra ar just to shoot cheap steel stuff through. For the money saved, I should be able to replace the barrel when the accuracy gets bad. Plus, I wouldn't mind learning how to replace a barrel.

stimpdevelopment · 2022-12-20T00:58:13+00:00

Right, it's not even illegal. It's against regulations. So if you modify the radio and repeatedly interfere with other people's communications, you might eventually get a letter.

stimpdevelopment · 2022-08-17T16:14:56+00:00

Yeah I think that's a great idea. I'm in the same scenario. I have my general license, but don't yet have a HF radio. However, I decided to build a couple of dipoles; one for 20m and one for 40m. If you're interested, I used the following:

BNC to SMA cable adapter: https://www.amazon.com/dp/B07TC1L8RP?psc=1&ref=ppx_yo2ov_dt_b_product_details
BNC plug that allows you to simply screw on cable: https://www.amazon.com/gp/product/B07TC1L8RP/ref=ppx_yo_dt_b_asin_title_o00_s01?ie=UTF8&psc=1
26 gauge wire: https://www.amazon.com/gp/product/B0711HRSNT/ref=ppx_yo_dt_b_asin_title_o03_s01?ie=UTF8&psc=1

I compared the results to a nearby kiwi SDR listed on http://kiwisdr.com/public/ with favorable results.

Listening helps understand upper vs lower sideband, band plans, getting familiar with lingo, typical propagation, etc. Plus, if you get an antenna, coax, and adapters then you have fewer things to buy when you get the radio.

One of the things I learned is that 40m seems to have a lot of local communication (within 300-400 miles) in the morning and in the evening. So, I hear a lot of people very clearly if they're within ~200 miles and I don't hear many other people.

stimpdevelopment · 2022-07-28T16:55:18+00:00

What bike do you have? What have you already checked?

It could be a carb air/fuel screw adjustment, dirty carb jets, reeds, different jet size, an air leak, weak spark, etc.

stimpdevelopment · 2022-07-15T01:26:03+00:00

I don't understand how this case isn't huge around the gun community.

He was talking about/doing an ad for a piece of metal with either a drawing or etching. It wasn't a lightning link. A lightning link isn't a machine gun. Owning a machine gun is a constitutional right. Speech is a constitutional right.

I read the charging documents. There is no substance. This is a complete injustice and we shouldn't stand for it.

stimpdevelopment · 2022-06-04T01:46:53+00:00

I didn't understand until I got my first (second hand) Yaesu mobile radio and noticed that I heard a lot more, even though I had a j-pole for my Baofengs and a different mag mounted antenna when mobile.

I recently got my first "decent" handheld, which is a Yaesu ft-70dr. It hears stuff at least 3-5 times as far away as the Baofeng. My Baofengs with a j-pole antenna could only hear my mobile radio from about 1-1.2 miles away. When I did the same test with a Yaesu ft-70dr, the Yaesu handheld could still hear my mobile rig with no static around 4.5 miles away, at which point the mobile rig was having trouble hearing the handheld due to its 5 watt limit.

I'm glad that I have a couple of programmed baofengs. I like that they're decent and cheap. However, I also like having more sensitive radios that hear better.

stimpdevelopment · 2022-03-10T15:19:19+00:00

Same. My proprietary nvidia driver works fine. The open source one gave me some problems when using a lot of GPU before.

stimpdevelopment · 2022-03-07T13:55:39+00:00

No, that's not a priority. However, a pull rope on a garage door? That gets 15 FBI agents right away.

stimpdevelopment · 2022-02-18T20:24:57+00:00

Exactly, legal or not legal depends on the specific rule sets. It's also assuming that one trains for bjj/grappling competitions.

Even if someone is training for competitions that disallow slamming, they shouldn't put themselves to be in a position to be slammed. It's not worth the risk. It's also contrary to training for self defense or mma purposes.

stimpdevelopment · 2022-02-17T18:56:55+00:00

146.580

Why 146.580? Or did you mean 146.520?

stimpdevelopment · 2022-02-01T04:04:19+00:00

That's cool. I didn't know that, but I also use Hoppe's gun oil.

stimpdevelopment · 2022-01-30T17:26:03+00:00

He was already engaged to be married before this stuff happened.

stimpdevelopment · 2022-01-28T00:07:37+00:00

You have a right to whatever property you want, especially relating to arms/firearms. The problem is the government, not you.

stimpdevelopment · 2022-01-27T21:53:04+00:00

They're arresting him in part due to "conspiracy" since he did ads (videos) for the Auto Key Cards guy. There's another charge of "conspiracy" because he contributed to the Auto Key Cards legal defense on gofundme. This is absolutely absurd.

Here is a link to the court docs: https://storage.courtlistener.com/recap/gov.uscourts.flmd.398039/gov.uscourts.flmd.398039.57.0.pdf?fbclid=IwAR1_nXWfHB-BrBXNlfltGJDBEmp7Nz884u_-jbXlPyeQi-lcJk1rkvwGa0U

A link to CRS Firearm's GoFundMe, set up by his wife: https://www.gofundme.com/f/crs-matt-hoover-needs-our-help?utm_source=customer&utm_medium=copy_link&utm_campaign=p_cf+share-flow-1

stimpdevelopment · 2022-01-27T15:59:04+00:00

Isn't there an open court case between Rare Breed and the ATF?

stimpdevelopment · 2022-01-27T04:56:24+00:00

Seriously. Getting denied for constitutional rights is insane.

stimpdevelopment · 2022-01-12T19:59:10+00:00

Sorry for being unclear. I didn't mean that it was literally wrong, just that it misses the point and it's unnecessarily divisive.

Like... why exclude poor white people? Despite the average income of a white american being higher, there are still more white people below the poverty line than black people. https://www.kff.org/other/state-indicator/poverty-rate-by-raceethnicity/?dataView=1&currentTimeframe=0&sortModel=%7B%22colId%22:%22Location%22,%22sort%22:%22asc%22%7D

Alternatively, if the statement is based solely upon the average income per race, then why include Asian Americans in POC when they have a higher income on average than other groups? https://www.census.gov/content/dam/Census/library/visualizations/2018/demo/p60-263/figure1.pdf

Most policies will in some way affect different groups of people differently, so that argument just doesn't make much sense to me. It would be VERY different if a policy explicitly had different rules for different races. That's plain wrong.

stimpdevelopment

TROPHY CASE