DemonGoddess Posted April 21, 2008 Report Posted April 21, 2008 First of all, once again I'd like to thank those of you who've so kindly volunteered to help. It's going to make this task much easier. We'll be doing this a subdomain at a time, and a top level IN that subdomain at a time. Keep an eye on this post for when we're ready to move on the next section. What we will be doing: For each story that you will be looking at, you'll be looking at and then doing the following: Misplaced or not Check briefly the story content to be sure it's in the right subcategory and/or subdomain. If it's not, then post a topic in Misplaced Stories with a link to it, and then tell me where it actually should be. It'll be moved to a special category that will be created for these stories, to be reinserted later to the tables in the subdomain it belongs to. If it's simply in the wrong category in the same subdomain, I can move it immediately. Let me know which category / subcategory it actually it belongs to and there it will go. Trolls If the fiction is an obvious troll fic, then put a link to it in the Trolls subforum. That way, the archive mods are aware of it, and can deal with both the user and the ficcie. Glitched The other thing you'll want to do with each story, (multi page stories), is follow the page links and check and see if the author is linked to still in the chapter. There are many, many glitched stories, where there is author data everywhere, including in the first couple chapters, then all of a sudden the author link disappears. When this happens, the author can't even access the chapter. If you find something like this, post it in the Glitched Stories forum, and say which chapter or chapters are affected. The chapter can then be fixed so that it is once again linked to the author. At some point in 2003 (also 2005 & 2006), many of you may remember when the database was corrupted. This was as the result of database crashes, and the glitches you see are in what was able to be restored, but was corrupted from the crash. Those will be stories with consistent weird wording errors, like "pounpounding" and other words you see like that. Post those in a separate topic, per story, with reference as to which chapter the glitch is in. Those actually will end up having to be edited directly in the table, provided that the chapter data isn't so long the db won't allow it. But, at least those that I can fix, will be fixed. Orphan Stories Orphan stories are stories which have NO author data attached to them. Generally those are caused by the user having been deleted at some point, usually for a TOS violation, but the fiction still remains in the archive. Those need to be deleted. So, again, links to the stories, and then all the tables with associated data from that story will have it removed. I don't just delete a story, I also have to delete chapters, reviews, and ratings. Those are four separate tables. 'un' Stories If the story content is something like an RP, RP request, challenge/request, beta request, searching for a fic; put a link to it in the "un" stories forum. Those will ALL be deleted. Duplicate Stories When looking in a category page listing, you'll sometimes see a duplicate story. There's quite a lot of that throughout the archive. With those, provide a link to each story, and then I'll be merging the data from one to the other, so the other keeps all the reviews, the ratings will increased exponentially according to the data in the other story, and the VIEWS will be increased according to what is there for the story data being merged. Once that's all done, the duplicate story will then be deleted. Of course, as long as it's all where it's supposed to be, and there's nothing else weird about it, that needs extra attention? Move on to the next story in the subcategory you're working on. Review Boards Next, we'll be eyeballing the review boards. Because some authors allow anonymous reviews, the software simply has no way to combat spambots. So, should you see spam posts in a review board, provide a link to that board in the Spam Activity forum, and reference the review number or numbers. Those will be deleted. If you spot a flamer in the user's review board, post that in forum where one would report a TOS violation. Again, the archive mods will see to it that it gets handled appropriately. We will be starting in the BOOKS subdomain, top level category A through F. Please post in response to this topic, which of the subcategories you'll be taking on. That way, there isn't duplication of work. Also, once you're done with the subcategories you've selected, if there are more available to work, you can choose more, or wait 'til we get to the next top level category. That's entirely up to you, and how much time you honestly have for this. None of you should feel obligated to take on more than your real life permits you, so please keep that in mind when selecting your work categories. Happy hunting!
DemonGoddess Posted July 14, 2008 Author Report Posted July 14, 2008 Books subdomain is complete. Two new top level categories, Anita Blake and Vampire Chronicles. We will be monitoring, and as needed, adding specific pair and other type sub subs for these top levels. 3 sub categories further categorized to the third level as they have 75 or more stories- Phantom of the Opera Peter Pan Sherlock Holmes We have found the errors, glitches and what have you for this subdomain, and they have been repaired. Also, manta2g is in the process of moving stories from this subdomain to other subdomains where they actually belong.
DemonGoddess Posted July 14, 2008 Author Report Posted July 14, 2008 Cleanup crew is now working in the Buffy subdomain. General is complete for sort.
drivven Posted July 21, 2008 Report Posted July 21, 2008 I've been trying to get to this and sadly been a lazyass. Sorry. I would like to help with the clean up project. If you'll let me know what and where, I'd appreciate it. Can give you my e-mail if you'd like. Thanks Cleanup crew is now working in the Buffy subdomain. General is complete for sort.
DemonGoddess Posted August 6, 2008 Author Report Posted August 6, 2008 We have finished in the AU and Crossovers sub categories for the buffy subdomain. Currently working in Het>Buffy/Spike(William)...we're approximately 2/3 done with sorting of that sub sub category. We will then move UP a level to Het and sort it, as the other pair specific sub subs did not exist prior to the sort in the General sub category. So, we've already eye balled them. In the process, we've also been looking for double and/or cross posted stories. For old ones, I'm simply merging the data. Where the cross post is in two different subdomains, the reviews for the deleted double will be lost. The hits and ratings will still be merged, in those for instances. We've actually only got about 1000 more stories to sort, and we're done with this subdomain. Shouldn't take too much longer.
DemonGoddess Posted November 22, 2008 Author Report Posted November 22, 2008 Sorry, but this has slowed down to a crawl lately. There are several reasons for that. First, is the need for active moderation of the archive itself, and getting that going again. That's a higher priority. Second, is the fact that of the volunteers I have for this, I have one with computer problems, one who is overloaded with schoolwork, and another who is overloaded with work and rl stuff. So, that leaves me and one other actively doing this at the moment. Last, but certainly not least, I'm having to fix what the upgrade messed up in the forum. As I work full time, this means that the major mods that don't work, I can only work on repairing on my off days. As well as my own chores at home, and other real life commitments I have. ugh. As I also have many other things on my plate at present, that means I cannot focus strictly on this, much as I would like to. However, doesn't mean I've been ignoring it, just means we're going slow. So, we're still in Buffy, but the finalization of sort for AU and Crossovers is complete with some changes which will apply in all the other single fandom subdomains as well. Hopefully, these changes will help people to find things easier, and compensate for the search engine. Which, as has been mentioned before, is simply not capable of handling a database of this size. It was never intended to. So, what that means, is that I try for more visual ways and means. In AU, the changes are as follows: It is now a top level category It now has as 2nd level categories, pair types Where, in the pair type second level categories, we have pair specific categories, the same rules apply here. A pair with 10 or more stories gets its own 3rd level category. Link to Buffy AU Click around, you'll see what has been done to date. Also, as we finalize the sort elsewhere in this subdomain, I expect the amount of 3rd level categories to grow. Changes in Crossovers It is now a top level category As 2nd level categories, it has pair types, as well as crossover worlds (5 or more stories written with that crossover). In some of the more heavily populated subdomains, I'm expecting I may have to go with two Crossover top level categories. I'm not sure yet, will know when we get there and see what's there, exactly. Third level categories are set up by pair type, in the world sub category where there are 75 or more stories, just as with the multi fandom subdomains. Link to Buffy Crossovers Just as with AU, as we continue to finalize the rest of the subdomain, I expect the number of 2nd level world categories to grow. Currently, only the Angel 2nd level category has enough to qualify for the 3rd level categories. That could change, depending upon what else we find in the sort and catalog. You'll notice there's a second level category called "General-misc". That would be stories where there is no pair type stuff, and stories that I flat out didn't know what to do with them, as the content did not allow me to determine where they belonged, exactly. In any event, this will hopefully make it easier to find things.
DemonGoddess Posted October 4, 2009 Author Report Posted October 4, 2009 After re-evaluating some things, we've decided to lower the required number of stories for pair specific sub sub categories from ten, to 5. Almost done with buffy, finally.
DemonGoddess Posted February 12, 2011 Author Report Posted February 12, 2011 When I am mass moving stories, users will see empty chapters and reviews. Ignore that please. It's a temporary thing. The export/import/transfer is done in a few steps.
DemonGoddess Posted January 28, 2012 Author Report Posted January 28, 2012 Meddy 2012-01-28 id # 3000030576 The new clean-up and categorization drew my attention on the fact that there is no "parent folder" link (nor a breadcrumbs bar) in the categories. With that many small categories created, it has become problematic. This is actually being addressed in the rewrite of the software itself.
DemonGoddess Posted February 2, 2012 Author Report Posted February 2, 2012 As I have 456498 chapters records to check for wallotext issues, the repair part of clean up is on hold while I find those and fix them. What this will entail, initially, is my placing these records in paragraph containers and reinserting in to the database. of course this does NOT affect all those records, but enough of 'em that it's going to take me a solid month or two to go through, find, and unwall. I'm not fixing any kind of errors this time around while fixing whether or not these things are in paragraph containers. Nor am I adding the formatting that may already be there. I'll get to all that later as we get to those various sub domains for actual complete repair
Guest Liule Posted February 5, 2012 Report Posted February 5, 2012 I just wanted to say that you're doing an awesome job, a horse job as it surely is, and that I'm (along with probably everybody else here on AFF) very grateful for the time and effort you're putting into this community. A wholehearted Thank You is definitely in place!
JayDee Posted February 5, 2012 Report Posted February 5, 2012 As I have 456498 chapters records to check for wallotext issues, 456498? 456498?? Holy fuck. Thank you for the time and the effort and, well, good luck with that!
Guest Walkabout Tigger Posted April 29, 2012 Report Posted April 29, 2012 I hope you're writing a script to deal with that!
DemonGoddess Posted April 29, 2012 Author Report Posted April 29, 2012 The problems with a script are 1. query overload 2. likely interference with the new code 3. The probability of missing other errors besides it that I have to go in and fix in the chapters individually ANYWAY
Recommended Posts