What are you migrating over?

So, I think that a lot of us right now in the early userbase are probably migrating over from DB, or might be exploring around after spending time in other boorus.

Are you planning on starting fresh here, or are you thinking of manually copying over your filters and watched tags list from your old account? How about your username or profile picture?

Personally, I think it's a good idea to start with a new username and password for every site or forum. Just basic internet security practice, really. I also made a new email specifically for this site, because I have a provider that lets me make shell addresses for signing up to newsletters as a security feature in case those emails end up being sold to some sort of third party.

I looked at the filters I was using back on my old account, but I noticed that there's very little of the stuff I was filtering on the old site on this new site. I don't know whether that's just because it's taking so long to upload all the images, or because these images are generally so low on everyone's priority list that they're not being transferred over as much as the other stuff.

I haven't gone through and added a ton of my "watched" tags. I don't know whether I should or not. It took a while to build up that watch list, but I'm not even sure if those tags are even populated or not on this new site.


TL;DR — I'm partially migrating stuff over here, but I'm not sure if I want to go all the way and copy all the rest of my settings. I want to see other people's decisions and reasoning on this.

I migrated all my favorite images and uploads over to this site, as well as some additional images that caught my eye. I used to have complex filters, but now they're very basic (everything, not safe, and safe + seizure warnings/watermarks), so there's no point of importing them. My watched images list is a mess of random things I remembered to add.

Now I'm going to focus on importing all my favorite images to Ponybooru. I want to have them on both sites just in case. After that, I'll prolly add whatever. Some from my favorite tags, from my favorite artists. I think focusing on older images is a good thing as they'd be harder to recover in case something happens to DB. A lot of those old sources are dead. I also want to focus on artists that removed their images from DB, such as senseidezzy.

How did you import them?

I've seen a couple of mass-downloading tools, but they seem to only be able to extract the tags and metadata in an external file, either as a json file (from derpibooru-downloader) or a different file that I can name and format any way I want (imgbord-grabber). Both programs are on github.

Is there a way to download the images plus metadata and them straight up upload them to the boorus?

I assume there's a way to mass upload these mass-downloaded files, but I haven't looked in to it, since what I had was thousands of unsorted or somewhat sorted files I've collected over the years.

I see the thread that links to the philomena image importer, but the original thread with the up-to-date version is down with ponybooru at the moment, and I don't see a github link to it.

The fact that there are so many backups and archives around right now makes me happy.

https://github.com/marktaiwan/Philomena-Importer

Lotus has uploaded over 40k so far so there must be a faster way to do it (then again, could be a way only admin has access to), but I've been using this and manually copypasting each link.

@ToastedTruffles

@phoenixFromTheTrashes
It's probably a very good time to announce this on the site. We are in the process of trying to download the Romesilvanius Derpibooru archive on a different server, and see if we can install the entire Derpbooru archive that way. I'm not entirely certain if we can actually make everything download correctly, if our script to change the data into pages will work, or if it will take months to generate all of the images, but that is our intent, and most of our focus on development is going towards making that happen right now. We are not entirely sure what will happen with all of the images that are uploaded now, but as we have two large servers plus backups running this there is no essential reason it will be lost. Please mark all images that don't exist on Derpibooru as such so it will be easier to transfer them and save them. There's probably going to be either a loss in saved favorites images or duplicates, whichever way it is done.

This is going to take a while, so you can keep uploading to keep the activity going, and besides, we may fail. But be advised

@Background Pony #ACD4
Sorry, I was supposed to make this post logged in.


@phoenixFromTheTrashes
I use the bulk importer script. It is extremely effective at scraping tags you like and automatically uploading them.
https://mega.nz/folder/aNwmzCaL#iQm2wdIYJL7V-zY3RtesVQ

@Lotus
I see. I was sure people weren't uploading tens of thousands of images by hand. There's not a lot of point, not with the tags and score information already existing from other sites.

I briefly checked out that tool you linked. From what I can tell, we ignore the dxwebsetup.exe and just launch nw.exe. It looks like the program automatically transfers all images from certain tag combinations from DB to either of the two other philomena-based boorus, with no offline download happening at all. Do I understand that right?

@ToastedTruffles
Yes. That is exactly right (except for Rainbooru being philomena based). It automatically uploads them after you fetch the tags you want, set an upload timer, and enter your API

Sweet. I was thinking about saving as much as possible for archival purposes, but now I think I'll keep uploading for fun and to keep the activity going. If there needs to be dupes I don't want there too many.

I'm assuming we mark non-derpibooru images with "ponerpics exclusive." Is this ok to put even if it's on a different booru? Or should they be marked with something like "not on derpibooru?"

@phoenixFromTheTrashes
Make up your own tags if need be so long as you inform me later. Right now I am using "ponerpics exclusive" to mean art first uploaded here (and not any other booru), and "not on derpibooru" to mean, well, it's not on derpibooru. You can also use "altbooru exclusive" if it's on ponybooru or rainbooru.

"Not on derpibooru" seems good. I was using "altbooru exclusive," but that might be misleading if some of these non-derpi images can be found on non-boorus, such as Twitter.

Idk if you already saw this, but if you're going to be mass importing, you might want to blacklist these tags and any others that might not apply upon importing: adventure in the comments, changelings in the comments, clopfic in the comments, comments locked down, comments more entertaining, debate in the comments, derail in the comments, discussion in the comments, duckery in the comments, featured image, index get, politics in the comments, shipping war in the comments, song in the comments, story in the comments, translated in the comments.

@phoenixFromTheTrashes
>changelings in the comments
I'm going to miss that.

Yes, we will do that. Thank you for giving a comprehensive list.

@Lotus
What did that mean, anyway?

@ToastedTruffles
It meant that there were many comments on the image by people with Changeling avatars. It was pretty common on images where the subject was one or more changelings.

@Lotus
Huh. Didn't know bugfags were that significant of a group.

So, I have a question that's tangentially related to the bulk importer script you gave a mega link to: How people get a list of the image IDs for the images in my account's favourites or upvotes?

I have a workaround for myself, but it uses no less than 3 programs, one of which is paid software:
1. I use the Derpibooru downloader,
2. download all the images from my:faves,
3. save only the Json file,
4. save the Json to a new folder,
5. set the name format for the Json file to {id} (meaning I delete the ".json" extension after the {id} that shows up by default
6. Follow the instructions in my file manager here [link] to print a list of all the files in a directory.
7. Realize that I screwed up, then go back and exit the format of the printed file list and make it so it's just a listing of the name of the file, ignoring the extension, file size, and other date.
8. Print the list to the clipboard, then paste the clipboard list into the Derpibooru bulk importer.

Edit: I assume the Derpibooru Bulk Importer program or the boorus themselves has some way to check whether the image file is already on there, right? Because I'd hate to import over ten thousand images and have half of them be duplicate files on the site.

@ToastedTruffles
Even this sounds more advanced that what I've been doing. I've been using an older version of the mass importer from before it had the black list. Bascially, I would look up "scenery," since it was the best tag to discern artistic talent, click on an image, click on the artist tag to see whatelse the artist does, if I liked it I would add the artist to the derpibooru importer, strip the links, and then add "AND NOT" to the link fetcher so that the links I fetched would not be redundant. With theblacklist feature on the newer version I would add the tags already striped to it. The reason I used artist tags is because the importer wasn't taking in more than around about the first 750 links on a tag, and artist tags very rarely have more than 750.

The bulk importer just basically applies the link to the importer. The site automatically rejects duplicates.

One problem I'm seeing, that I noticed when I was trying to mass download DB files the other day, is that canon character names aren't recognized as character tags. There's the artist tag, a tag for the rating, but it seems that only fandom OCs have character tags.

Edit: Update
So, I tried to let the mass importer program running on my computer overnight, but I think it crashed at some point. I don't know why. I think my computer must have gone into sleep mode or something. I clicked the importer's entry in my taskbar, but it disappeared and couldn't be found again.

According to my upload logs,I've only uploaded 6.9k images, so it was definitely stopped partway through.

My workaround is that I'm setting it up to upload again, but in reverse this time. That'll definitely screw with the upload order, but importing an image that has the Derpi ID of 2297729 with that tool will give it that same ID on this site, right? It won't re-number the images all over again, I hope. Otherwise I just fucked things up a lot by uploading selections instead of uploading the images in the same order they were uploaded to DB.

@ToastedTruffles
I should mention that after you mentioned the tags, I went through and manually changed that so at least the most popular canon characters have the colored character tag.

As for the links and import, I am not sure. I have experienced similar problems in the past. I think it stops or crashes when the computer goes to sleep, but I am not sure.

The weird thing for me is that there were some remnants of the program running when I logged back in to my computer. I'll just chalk it up to some weird program behaviour when it's interrupted. I don't think these tools have been under years of continuous development or something, so I can't hold it against the devs. They do the job well enough for most purposes.

Oh well. I figured out that my computer was going to sleep mode 4 hours after I'd last interacted with it. Normally a perfectly reasonable setting, but not for this. I learned how to disable it for tonight.

Thanks for adding the character tags.

Now the only tool I need is a "mass favouriter" and "mass upvoter", assuming the image IDs are still preserved.

I'd say the Booru migration has been going pretty well, all told.

@ToastedTruffles
Never really had an account back on the old DB, planned on actually staying despite the shitshow. But seeing how everyone is raving about "The End of Derpibooru", I might just make an account to stay here, hopefully to see more ponies as Derpibooru becomes the centre of scrutiny and chaos

@Background Pony #E9CE
We certainly hope you'll stay here. It should be easier and more fruitful after we have the whole derpibooru catalog and a few quality of life improvements.

@Lotus
Well I'm deciding to stay now since I've gone around trying and seeing this as a much more better site than most other alternatives.

I made my account earlier today after growing concerns over what is happening over at Derpibooru, and in addition, finding out that one of my posts in the "ask the mods anything" thread, Which simply questioned site mod Derpy Whooves's claim and the full validity of the Oxford study concerning Russians doing psyops on social media. I didn't make any death threats nor throw slurs at anyone, just asked questions concerning the study. The reason stated for deletion was "Rule #6- attempting to derail". How considerate and totally not biased.

@ToastedTruffles
I'll bring over my watched, filters, and favorites (if I can get the Booru Sync script to work). Will probably pick a new username. Not sure whether I'll pick Horsefucker101, FistMare, or Big Gay Jackrabbit. Which MySpace pic I use over here depends on the username I select.

Derpibooru is a horrific place, and I am looking for somewhere less horrific.