Future possible plans for structure

[DemonGoddess]

Okay, I've been talking to some people who write in this fandom, for when we have the software written capable of supporting what this subdomain needs, so it can be done in such a way as to not overload the database server.

One of the biggest suggestions I've gotten is when we get to that point, where I am no longer restricted to 3 levels of categories, is to make (under pairing type, etc), a category that is character specific. Then, as a sub of THAT sub category, would be the specific pairs.

So, for example, where now you have HP>Het>Hermion/Ron, it would be HP>Het>Hermione>Hermione/Ron. And so on.

I can't do this now, of course, as the current software simply won't allow it.

Thoughts?

[madlodger]

Okay, I've been talking to some people who write in this fandom, for when we have the software written capable of supporting what this subdomain needs, so it can be done in such a way as to not overload the database server.

One of the biggest suggestions I've gotten is when we get to that point, where I am no longer restricted to 3 levels of categories, is to make (under pairing type, etc), a category that is character specific. Then, as a sub of THAT sub category, would be the specific pairs.

So, for example, where now you have HP>Het>Hermion/Ron, it would be HP>Het>Hermione>Hermione/Ron. And so on.

I can't do this now, of course, as the current software simply won't allow it.

Thoughts?

It's a good idea. I like Harry-centric fics, for example, and this would be helpful. On another hand, we are going to end up with this:

HP>Het>Hermione>Hermione/Ron

HP>Het>Ron>Hermione/Ron

Or even more for threesomes and such.

So, the same fic would belong in more than one subsubcat. This could work if the fics can be assigned to multiple categories and appear on the "Latest Updated" pages in all.

[DemonGoddess]

that depends on the coding and what we can do with it, while keeping the database intact. Everything hinges on being able to preserve the old data as well as integrating it into a new software structure. Personally, I find it easier to just go ahead and alpha sort stuff anyway, as far as category names. But that's a "thing" for me.

But, what I'm thinking as well, is because we'll have a new very strong search engine in place when this happens, is that the category names will work as search tags as well. I don't particularly care for multiple categories covering the same pair, kind of redundant. Unless, of course, it's because it belongs AU/AR, Crossovers, or something like that. That's a bit different. Then again, we'll see how it all it falls together once we get to that sub domain for sorting in the first place. It's hard to say exactly HOW it'll all come together, until we're actually working there and really looking at all the data that is there, and how it can be made to work, both with current structure, and for future structure. Because that's also part of it, setting up as much as possible before the software is changed, so it's not difficult to then move the stuff around to where it'll ultimately end up.

Now, going back to character-centric type stuff, that is the ONE thing that kept coming up when I was asking various writers opinions before I made the initial post. And I can also see that by doing it that way, it'll make it easier to do something about the OFC/OMC pair with canon characters stuff, in such a way so as to not overtax the database and/or cause instability in the database. That's the big thing, you know, keeping the database itself stable. With an unstable database, we have no archive.

So, as we work in each area, certain things become apparent when sorting that are going to end up unique to the sub domain we're in, such as character centric in this one in the future.

The threesome stuff will remain as it is, I think, and when we get to sorting it, where it's warranted, a specific threesome+ will have it's own sub sub of the threesome subcategory.

You can actually get a good idea of much of it, if you look at the buffy sub domain. One of the things we've been looking at there, and very seriously considering, for example, is moving all the Angel fanfiction from the tv sub domain to there, as Angel is a spinoff series. But, until we figure out a way to do it across tables and sub domains without losing or corrupting data, it's going to stay the way it is.

Latest updated, or new link shows what's new in the entire subdomain. It has never sorted that list by category or subcategory, but by date and time of update. Whether a new story, or a new chapter. Edits don't count, of course.

[madlodger]

that depends on the coding and what we can do with it, while keeping the database intact. Everything hinges on being able to preserve the old data as well as integrating it into a new software structure. Personally, I find it easier to just go ahead and alpha sort stuff anyway, as far as category names. But that's a "thing" for me.

But, what I'm thinking as well, is because we'll have a new very strong search engine in place when this happens, is that the category names will work as search tags as well. I don't particularly care for multiple categories covering the same pair, kind of redundant. Unless, of course, it's because it belongs AU/AR, Crossovers, or something like that. That's a bit different. Then again, we'll see how it all it falls together once we get to that sub domain for sorting in the first place. It's hard to say exactly HOW it'll all come together, until we're actually working there and really looking at all the data that is there, and how it can be made to work, both with current structure, and for future structure. Because that's also part of it, setting up as much as possible before the software is changed, so it's not difficult to then move the stuff around to where it'll ultimately end up.

Now, going back to character-centric type stuff, that is the ONE thing that kept coming up when I was asking various writers opinions before I made the initial post. And I can also see that by doing it that way, it'll make it easier to do something about the OFC/OMC pair with canon characters stuff, in such a way so as to not overtax the database and/or cause instability in the database. That's the big thing, you know, keeping the database itself stable. With an unstable database, we have no archive.

So, as we work in each area, certain things become apparent when sorting that are going to end up unique to the sub domain we're in, such as character centric in this one in the future.

The threesome stuff will remain as it is, I think, and when we get to sorting it, where it's warranted, a specific threesome+ will have it's own sub sub of the threesome subcategory.

You can actually get a good idea of much of it, if you look at the buffy sub domain. One of the things we've been looking at there, and very seriously considering, for example, is moving all the Angel fanfiction from the tv sub domain to there, as Angel is a spinoff series. But, until we figure out a way to do it across tables and sub domains without losing or corrupting data, it's going to stay the way it is.

Latest updated, or new link shows what's new in the entire subdomain. It has never sorted that list by category or subcategory, but by date and time of update. Whether a new story, or a new chapter. Edits don't count, of course.

Well, I know that AFF is backing up data every night. In my mind, I see a box full of shiny disks going back since the inception. So, if the attempt to transfer data had failed, the site could just go back and restore to a day before. The new software would be in beta after it actually comes out, right? To iron out all potential bugs before turning it into main.

Right now the structure looks like a collection of separate web sites with a central control panel thingy. I can see how it would be difficult to move data from one site to another, but technically, some custom data transfer program should be able to do it. I mean, as opposed to moving data by hand, which is also achievable, even though very time consuming. I am not a programmer, though, so can't help here.

About things like a Buffy site: Why not create a poll about this and invite Buffy/Angel peeps to participate? Tell them upfront that this is for new software development, and the results of the poll(s) will be implemented if feasable, and not if impossible from programming stand point. This is all in the idea generating stage still, right?

Last updated right now is the feature I use a lot. It gives a good feel to what is happening in the community as a whole and I found some awesome fics this way in the subs I don't usually check.

[DemonGoddess]

yes it gets backed up nightly, as part of the hosting package. BUT, what I'm talking about is potentially corrupting the tables themselves and really screwing it up, and not CATCHING it right away. That can easily happen when you're talking about transferring data around within tables. And, if that happens, and it's caught after the backup is replaced, it's replaced with bad data. A good example of that is this latest forum upgrade you know. I didn't find the cause of the problem (corrupted skins) until AFTER that 24 period was up. Restoring the old skin files would simply make things worse, so redoing the skins themselves is the only thing I can do about it. Would have to have been done in any case.

However, before doing something like that, I'm in the habit of making sure I have all data available locally anyway, as is manta2g. So that we have a set to go back to. But, in that sort of for instance, then anything it's restored to, (say we didn't catch it for a week), that entire week's worth of data is gone.

Yes, we do indeed plan on testing software very thoroughly before unleashing it upon the archive at all. The idea is to have a rather seamless transition, with very little down time.

Actually, this site is set up with a series of sub domains. So what happens is that a copy of the program is installed to each and every sub domain, with a central point of some scripts being only at the root domain itself. There are also a series of tables which govern the archive per subdomain. That's where the difficulties come in, as the tables are identical in set up, and we have to find a way to append it without overwriting what someone else may be adding while we work. So that means we look for blank blocks in the record structure.

The split, or divide as it was referred to at the time, was done because the OS (LinuxBSD) did not allow anything bigger than a 2GB database. So, by doing this (all the sub domains), one could trick the OS into believing the database never exceeded that size, even though it already had. Which was the cause of the large crash 4 years ago. As you know by looking at those old stories, most of the data is horribly corrupted. Hence the clean up project, and earmarking and repairing the stories.

And, as to the moving of one subsection entirely, yes, before even doing something like that, I would've posted it in the archive and asked about it, rather than just doing it. It came to mind BECAUSE during the sorting, we (the other gals who help with that and I) noticed that there are quite a few Angel the Series stories uploaded to the Buffy sub domain, and STILL being uploaded to that sub domain. I expect once we get to tv and the Angel section, we'll see a similar pattern.

Again, it goes back to certain things that fit certain fandoms and would be unique to their particular sub domains, like the character centric thing here, and categorizing by the ages for LotR. As you can tell, we won't know 'til we actually GET there and take a good look at the data itself.

This sub domain is only a couple away from being worked, which is why I'm asking this now, even though much of it cannot be done yet. Even still, for cataloging purposes, it's good to know so we have it recorded for later, and can hopefully come up with something that might possibly work in the interim before the deeper sub leveling can be implemented. At that point, it then becomes a matter of moving sub categories around and the stories in them (for the most part), rather than moving individual stories, which is what happens this time around. Aside from that, there's also the matter of the current search engine which was never designed to handle a database this size. So, certain changes already made (like AU/AR and Crossovers becoming top level categories), simply make it easier on the USER to find things that the search engine will not find for them now. It only returns the first 100 results of any search, so if you search something that has 1000 results, you'll never see it all, unless you manually go THROUGH each and every story wherever they happen to be uploaded/added to.