One forum left.
09-11-2017, 08:05 AM (This post was last modified: 09-11-2017, 08:09 AM by Bob Schroeck.)
09-11-2017, 08:05 AM (This post was last modified: 09-11-2017, 08:09 AM by Bob Schroeck.)
<yawns> I was up about two hours later than I should have been last night, getting the last of "Other People's Fanfiction" from the initial scrape loaded. Between dealing with about 30 or so threads that failed because of the string concatenation problem (requiring me to back out the failed thread, find the missing content, reconstruct the insert statement, and re-run, followed in a few cases by repeating the process because I introduced a new error) and getting caught up in reading some of the oldest content when I should have been paying attention to the DB tasks, it took me a good 8 hours or so.
"General Chatter" has something like three times as many threads as "OPF" does. I'm warning people now that it's not going to happen in one night. (One good thing is that as far as I can recall, "General Chatter" is unlikely to have many posts in excess of 43.5K in size -- the point at which that string problem kicks in. As a result, I should have an overall faster throughput on the transfer. Fixing the failed inserts is the largest time overhead in the process.)
Oh, also, fair warning, folks: to minimize lost threads/posts, I'm going to lock the forum before I start the move (which is really how I should have done it for the other fora, but oh well...). So basically starting tonight, everything is going to be closed over on the old boards.
"General Chatter" has something like three times as many threads as "OPF" does. I'm warning people now that it's not going to happen in one night. (One good thing is that as far as I can recall, "General Chatter" is unlikely to have many posts in excess of 43.5K in size -- the point at which that string problem kicks in. As a result, I should have an overall faster throughput on the transfer. Fixing the failed inserts is the largest time overhead in the process.)
Oh, also, fair warning, folks: to minimize lost threads/posts, I'm going to lock the forum before I start the move (which is really how I should have done it for the other fora, but oh well...). So basically starting tonight, everything is going to be closed over on the old boards.
-- Bob
I have been Roland, Beowulf, Achilles, Gilgamesh, Clark Kent, Mary Sue, DJ Croft, Skysaber. I have been
called a hundred names and will be called a thousand more before the sun grows dim and cold....
I have been Roland, Beowulf, Achilles, Gilgamesh, Clark Kent, Mary Sue, DJ Croft, Skysaber. I have been
called a hundred names and will be called a thousand more before the sun grows dim and cold....