[library organization] How do you organize your stories

[fmk007]

Hello all,
since my repository of erotic/smut stories and books got really big I try to rethink the way it is organized.
Actually my collection is stored in folders / subfolders with the source on top (Literotica, AO3, Web ....) below genre-subfolders (TG, mc, inc, bi ...) where the stories are directly in with filenames in the form TITLE - AUTHOR.

This system did work for me for years (when the sources were few), but meanwhile I have added a lot of stories from many different sources.
Now the authors are scattered in many different folders and a lot of doublettes in different formats and styles as well.

I try now to reorganize all this stuff but have no clear vision how to do that. Either I make a better folder structure on the disc (which need a lot of individual edit) or I use some sort of existing database/organization solution like Calibre. (But more focus on library organization than reader-devices or converting)

Therefore I'm asking you, how do you organize your stuff to get ideas for my own situation

 

[fmk007]

My reason to post this in this forum:
Nearly all relevant sources are focused on digital editions of 'real' books and usage of different reader hardware.
My focus is not file format conversion or preparing for different reader hardware.
I simply(?) want to have my lib in an order which is easy to maintain to find stories easily (via author, title or tags) and without all the duplicates and broken stuff.
My collection has mostly stories or series from many sources with unknown authors or same story with different author aliases (or stolen) with a lot of different (overlapping) genres, which makes organizing a lot more complicated. And there are lots of broken/incomplete files/series.

 

[FC1(SW) Retired]

I organize my library by a combinition of zip files by "author (genre genre genre...)"
Until I finally load them into calibre library manager - https://manual.calibre-ebook.com/
Calibre is a free very versatile library manager, even going so far as to allowing you to customize some features (note: I would experiment adding the custom stuff in an experimental library because deleting fields you decide you don't actually want doesn't appear to be an option)
The only real drawback I've found with putting the books into calibre is that it truncates file names, but since it's a library manager you can still easily find them, it's just when I go to share I find I have to temporarily rename the file so it's fully identifiable (Title-author) as an upload, and then change it back so calibre can still find it.
I used to identify where I found them too, but after a couple site implosions like 8muses, I decided that it wasn't really worth the effort. Now I just regularly back up the library hard drive, so I won't need to go back and find it again,

 

[yano2mch]

For the moment i organize them in two methods. Raw authors (Books i got from Anne's and Z-lib), and then the thread's name [Author] Genre.

There is a third category of classics, where they are done by publisher, Adult Line Books, Beeline books, Centaur Press, etc. But that's all for 90's paperback smut with old covers. Though i'm finding a small treasure trove of older classics, it will take a while to sift through it all and getting 10 books at a time is slow.

 

[vkg123]

The simplest way is [Author] - series (if any) - title.
or [Genre] Author - series - title.

If there are a lot of standalone books or from a specific publisher or source then [Genre] can be swapped for that name.

 

[O_P_T]

My organization depends mostly on how I obtained the book and/or format

Any standard ebook format (epub, mobi, azw, etc.) I put in my calibre library.

These are books I get from Amazon, Smashwords, Mobilism, here, Z-library, Zoboko, etc.

Regarding stuff I've gotten from literotica, Lush Stories, Stories Online, Kemeno, etc. that depends on when I got it.

Today, I use FanFicFare to make an epub for stuff from literotica, that is a plug in for calibre. That plugin populates all the metadata when it downloads a story, and one can easily update it if the author adds new chapters. It also puts the URL for the story in the ids field and adds a blue arrow next to the author listing that is a hyperlink to their page at literotica.

I use EpubPressX ( a plug in for Chrome) to make an epub, and put that in calibre. It will put a "reference" section at the end with the URL for every web page used to make the epub.

I have to manually populate the metadata in calibre. I use the website name as the publisher (e.g. literotica, Stories Online, etc.) that makes it easy to sort the calibre library based on the source.

In the ids field, where normally are the links to the book at Amazon, Goodreads, Smashwords, etc. I put the URL for the authors web page where the material came from. The format in calibre is to type "url:" then paste the URL after the colon. One limitation for calibre, is that it will only show one url listing when you're browsing the "bookshelf". So, if a story is posted on more than one web site, you can't put both in the ids field. You can put ids for Amazon, etc. along with the url ids.

For some web pages, EpubPressX doesn't work, not sure why. So for those pages, I do what I did in the before time before I had FanFicFare and EpubPressX. Copy and paste into a Word document.

I go into the info tab and put the author's name, and sometimes populate the rest of the metadata fields.

Generally speaking, I put these Word documents in one folder, although I do have some subfolders. Some of those are based on the source of the material, others on the author.

These subfolders are typically used when there is a story that is still being updated, and it simplifies looking for a specific story when I'm at that site looking for updates.

Since I have the author names saved in the Word file metadata, it's pretty simple to do a search for any given author's name.

 

[yano2mch]

Any standard ebook format (epub, mobi, azw, etc.) I put in my calibre library.

I have a heavy preference for epubs, since they are zip files you can inspect or fix/change them compared to other formats. Unless conversion isn't an option or it's for print (like D&D PDF files where it's image heavy and mixed with text).

 

[O_P_T]

I have a heavy preference for epubs, since they are zip files you can inspect or fix/change them compared to other formats. Unless conversion isn't an option or it's for print (like D&D PDF files where it's image heavy and mixed with text).

Well, I standardized on epub for a different reason.

~2015 I had to travel a bit more for business, and I usually read on a plane, or while waiting at the airport. I got annoyed when I'd have 20 odd pages left in the book I was currently reading when I left for a trip, so I'd have to bring another book along with me. Or more than one book if the trip was going to be long enough.

Most of these business trips lasted long enough, so that I wouldn't have enough clean clothes if I did just carry on, so I usually checked a bag.

In those types of trips, I preferred to keep my carry on bag to be a small as possible, so I wouldn't have to carry a lot through the airport. I would limit it to the stuff I absolutely had to have with me when I got to my destination: medications, company PC, my cell phone, travel documents, etc. So, lumping multiple books in the bag was a relatively big impact on what I carried.

So, I bought an iPad. I made the decision that I wanted all the books on the iPad to be in a single application, since i didn't want to jump from one app to another to see all the books I had available to read.

I settled on iBooks, and so had to convert any mobi/azw, etc to that format to make that work.

 

[SmeeJ]

I prefer plain text. Back in the ASSTR days, I started collecting from there, and plain text just made sense at the time. I then wrote my own software to manage the files and associate metadata with them to make it easier to find stories again. Still prefer plain text to ePub even after discovering Calibre.

 

[fmk007]

My situation is mostly the same as described by O_P_T
A huge vcollection of older stuff resulting from 35 years of collecting.
Most of the stories are in TXT format, sometimes with a html header and therefore I have a lot of identical stories with different extensions (TXT,htm, html) and also different order of file names (title author or author-title ...).

Actually I change all the file names to follow one rule (Title( - Series) - Author - Publisher) to enable furthter cleanup.

Calibre is not suitable for this TXT file repository (with additional covers or other files), but it is perfect for my "real" eBooks (with ISBN)
I try to find an easy database solution to read in the directories (with a hyperlink for each entry would be perfect) to be able to sort through per Title / Author etc. and add some columns with metadata or other data of interest.
This would be a great help for cleanup at the first stage, but also should work as the entry point to find and read the stories. (similiar to Calibre, but with the files staying in their original position with their full filename - so the folder structure follows my rule instead Calibre rules)

Years ago I started this with Excel, but this is too slow for my big repository with serveral 100000 entries.


@Smeel: I would be interested in your script for adding metadata to TXT files (if it runs on Windows)

 

[SmeeJ]

@Smeel: I would be interested in your script for adding metadata to TXT files (if it runs on Windows)

Not scripts, a full fledge application written in Objective C. I used it as an opportunity to learn to code for iPhones when that seemed a gold rush. Not sure I can really release it.

 

[KidneyCracker]

For stories from free websites:
SQL Express database maintained using an application I created. Can scrape stories based on story URL from a variety of sites. It can also create epubs of these stories or custom web pages that I can use MS Edge's free ReadAloud cloud narration to listen to. Sounds natural enough.

I don't bother with Calibre for epubs because I always found it too slow. Instead I just manually sort files in folders:

Publications

1-My Custom Books <- epubs of stories scraped from free online sites for mobile reading​

2-Epub Books (Read) <- one folder full of completed books​

3-Books to Store but Ignore <- one folder full of books by authors I didn't enjoy​

-Suspected Plagiarism​

4-Epub Books <- This folder tree gets synced to my phone. only contains unfinished books​

-A <- First letter of Author's Given Name​

[Author Name] - [Series Name, The] <series number> - <book name if applicable, The>​

-B​

...​

-Z​

 

[O_P_T]

For stories from free websites:
SQL Express database maintained using an application I created. Can scrape stories based on story URL from a variety of sites. It can also create epubs of these stories or custom web pages that I can use MS Edge's free ReadAloud cloud narration to listen to. Sounds natural enough.

I don't bother with Calibre for epubs because I always found it too slow. Instead I just manually sort files in folders:

Publications

1-My Custom Books <- epubs of stories scraped from free online sites for mobile reading​

2-Epub Books (Read) <- one folder full of completed books​

3-Books to Store but Ignore <- one folder full of books by authors I didn't enjoy​

-Suspected Plagiarism​

4-Epub Books <- This folder tree gets synced to my phone. only contains unfinished books​

-A <- First letter of Author's Given Name​

[Author Name] - [Series Name, The] <series number> - <book name if applicable, The>​

-B​

...​

-Z​

IIRC, you stated in a previous post that your SQL Express program had an issue with stuff posted at StoriesOnline because one had to "click for more" to get the full page loaded.

I don't know if you noticed, but I posted somewhere, that they seem to have removed that requirement.

Every story I've grabbed over the past few months, using EpubPressX, had the full content of each story/chapter, when I opened it in a tab.

 

[KidneyCracker]

Thanks. I haven't been digging through that site much lately to notice if it has changed. They had a timed restriction on stories, so anyone can read them for a fixed set of time, then they go behind the paywall for full access. Maybe the "click for more" is another stage of timed access? Used to be all over the site.

I've only looked at a few newer stories recently and I am really only interested in Infinite Eleven's new posts. The site is a ghost-town for content I like/authors I recognize.

 

[O_P_T]

Thanks. I haven't been digging through that site much lately to notice if it has changed. They had a timed restriction on stories, so anyone can read them for a fixed set of time, then they go behind the paywall for full access. Maybe the "click for more" is another stage of timed access? Used to be all over the site.

I've only looked at a few newer stories recently and I am really only interested in Infinite Eleven's new posts. The site is a ghost-town for content I like/authors I recognize.

As I recall, next to the "click for more" there was a "why am I seeing this" and it said they did this to prevent bots from copying the stories, or something to that effect.

 

[KidneyCracker]

As I recall, next to the "click for more" there was a "why am I seeing this" and it said they did this to prevent bots from copying the stories, or something to that effect.

that would be me. Although I don't have any nefarious plans to plagiarize, repost or re-sell them like some bots. I just want a copy in case the stories disappear before I read them.

 

[midir]

My organization depends mostly on how I obtained the book and/or format

Any standard ebook format (epub, mobi, azw, etc.) I put in my calibre library.

These are books I get from Amazon, Smashwords, Mobilism, here, Z-library, Zoboko, etc.

Regarding stuff I've gotten from literotica, Lush Stories, Stories Online, Kemeno, etc. that depends on when I got it.

Today, I use FanFicFare to make an epub for stuff from literotica, that is a plug in for calibre. That plugin populates all the metadata when it downloads a story, and one can easily update it if the author adds new chapters. It also puts the URL for the story in the ids field and adds a blue arrow next to the author listing that is a hyperlink to their page at literotica.

I use EpubPressX ( a plug in for Chrome) to make an epub, and put that in calibre. It will put a "reference" section at the end with the URL for every web page used to make the epub.

I have to manually populate the metadata in calibre. I use the website name as the publisher (e.g. literotica, Stories Online, etc.) that makes it easy to sort the calibre library based on the source.

In the ids field, where normally are the links to the book at Amazon, Goodreads, Smashwords, etc. I put the URL for the authors web page where the material came from. The format in calibre is to type "url:" then paste the URL after the colon. One limitation for calibre, is that it will only show one url listing when you're browsing the "bookshelf". So, if a story is posted on more than one web site, you can't put both in the ids field. You can put ids for Amazon, etc. along with the url ids.

For some web pages, EpubPressX doesn't work, not sure why. So for those pages, I do what I did in the before time before I had FanFicFare and EpubPressX. Copy and paste into a Word document.

I go into the info tab and put the author's name, and sometimes populate the rest of the metadata fields.

Generally speaking, I put these Word documents in one folder, although I do have some subfolders. Some of those are based on the source of the material, others on the author.

These subfolders are typically used when there is a story that is still being updated, and it simplifies looking for a specific story when I'm at that site looking for updates.

Since I have the author names saved in the Word file metadata, it's pretty simple to do a search for any given author's name.

Is there any way to batch download all the stories of a author in litertoica?There used to be a tampermonkey script but it stopped working

 

[7hR337R0723]

Is there any way to batch download all the stories of a author in litertoica?There used to be a tampermonkey script but it stopped working

The Linkclump Plus chrome extension + FanFicFare Calibre extension work pretty well.

Set up Linkclump Plus to copy URLs only (not link text) to clipboard (the defaults in Linkclump Plus are to open selected links in new tabs, and to copy the link + text - you have to change this in the extension's settings). Then just hold "z" down and select all links to copy the URLs.

In Calibre, open up FanFicFare > Download From URLs, then paste. It pastes all the story links and they are downloaded and converted to epubs in one go.

To handle situations like stories with many parts, FanFicFare seems to be able to de-duplicate so no additional work needed. Need to be careful not to overwrite metadata for existing stories you already with FanFicFare (if you made annotations).

 

[qwerdy0]

This is a great topic. I struggled with categorizing/tagging stories for years. Still never really figured out a good way to do it with close to 400K stories I've collected. I did find this project on github which has been very helpful. https://github.com/sist2app/sist2

It uses elasticsearch to index folders (.epub, .pdf, .txt., .html, etc) and then has a front end to search with. It's the best thing I've found so far. I want to eventually use a local LLM to summarize and tag all my files (maybe in Postgres), but I just haven't had time to work on it.

Is there any way to batch download all the stories of a author in litertoica?There used to be a tampermonkey script but it stopped working

Another great github project is litstash which uses pyhton to grab stories from literotica and put them into html files. You can also put in an author's name and it grab all their works, or put in individual stories. https://github.com/NocturnalNebula/litstash

There's some great stuff in the open source world.

 

[O_P_T]

The Linkclump Plus chrome extension + FanFicFare Calibre extension work pretty well.

Set up Linkclump Plus to copy URLs only (not link text) to clipboard (the defaults in Linkclump Plus are to open selected links in new tabs, and to copy the link + text - you have to change this in the extension's settings). Then just hold "z" down and select all links to copy the URLs.

In Calibre, open up FanFicFare > Download From URLs, then paste. It pastes all the story links and they are downloaded and converted to epubs in one go.

To handle situations like stories with many parts, FanFicFare seems to be able to de-duplicate so no additional work needed. Need to be careful not to overwrite metadata for existing stories you already with FanFicFare (if you made annotations).

One issue I've noted with FanFicFare, at least as literotica is concerned, is that it doesn't always seem to link the various chapters together. I've seen instances where there's a chapter nested into a series, but FanFicFare doesn't see it as part of that series, either with a new download or an update.

I've also seen cases where a new chapter wasn't shown nested in a series but FanFicFare did add it, again either in a new download or update.

What I thought was happening, is that FanFareFic looks for a link to the next chapter in a series that normally appears on the last page of any chapter under the "read more of this series" banner.

I've noticed in the past that that banner doesn't always contain all chapters of any given story, especially if it was posted recently to the time you're viewing it.

However, I just checked one series and it doesn't have such a banner, yet FanFareFic grabbed all the chapters.