KRAFTIES STATUS

🟢

Krafties is up!

All systems are operational!

Having trouble? Let us know in the Krafties Official group chat!

Additionally, it may help to check the status of some of our service providers. Issues listed on their status pages can often times impact Krafties in various ways:

OLDER STATUS UPDATES FOLLOW

MAY 2026 RECOVERY UPDATES

Update #14 From the Dev (May 10, 2026 12:30AM):

Tomorrow it all happens.

This will be the final update here.

Thank you for reading.

Update #13 From the Dev (May 9, 2026 2:30AM):

Everything looks like it’s working well!

There was only one small incident today, which was caused by an intentionally lower resource allotment assigned to the new database server in an attempt to see if the monthly cost could be reduced with our new configuration. Turns out, it can’t, but that’s ok. A quick tweak to the VM and we were back in action.

Other than that, things seem to be going eerily smoothly… which meant that I could put some more time into design work for a special something I’d like to do for players to commemorate this glorious occasion. And to just rest. It’s nice to exist in a state that isn’t constant urgent dread. Really, really nice.

If things keep going like this, we’ll be looking at Krafties *officially* being online with an all-clear tomorrow or Sunday.

Update #12 From the Dev (May 8, 2026 1:00AM):

Annnnnnnnnnnnnd, we’re back!

It took a bit of finagling to make our existing application and caching servers play nicely with the new stuff I’ve thrown into our database setup (basically some layers of virtualization and containerization, which I’m hoping will allow for better maintenance and backup flows), but as of now things are not only connected, but seem to be FULLY FUNCTIONAL IN-GAME.

After seeing the first hundred accounts access the servers, producing battle and item logs without any errors, I feel safe letting myself believe that this nightmare might just be (almost) behind us.

There’s still a whole lot of work to do on this end, but I believe it’s work that can be done while the game runs, and may only result in a few minor hiccups as different servers take a few seconds to reboot every now and then when settings are changed. The big thing left to tackle is strengthening the connection between our application servers and the database. They’ve been more or less “manually” glued together for now, but we’ll need to make sure they automatically reconnect between reboots and updates, keep specific processes running without any manual intervention, etc. Also, watching every single little thing like a hawk to make sure that nothing else goes wrong…

BUT, I’ll pick that all back up tomorrow.

It’s been a good day 🙂

<3 <3 <3 <3 <3 <3 <3 <3

Update #11 From the Dev (May 7, 2026 2:30AM):

MUAHAHAHAHAHAHAHA

muahahaa

Thing’s look good.

I’m going to throw out two predictions that I’m at around 80-90% on right now

Krafties will be back up by the weekend
Our data will pick up where we left off nicely

muaha

Update #10 From the Dev (May 6, 2026 9:00PM):

Did I mention how slowly this backup file re-population was going? And now I’m all out of cookies.

File size comparison looks promising, though. Can’t wait to start comparing the actual data so we can finally, finally know whether all of this work has been worthwhile.

Maybe a few more hours for this database to be done rebuilding, then I can start doing stuff again.

Update #9 From the Dev (May 6, 2026 3:00PM):

An extra instance of the database is now being created from an older backup file so that we can ensure that the tables in our recovered, up-to-date, nicely restored instance of the database are 100% sound.

Golly gosh darn is it a friggin slog watching all of this data being repopulated, though… but we are oh so close to the finish line!

I think I’m going to get myself a cookie or two while this copy rebuilds. Yum.

Update #8 From the Dev (May 6, 2026 3:30AM):

All recovered files have been successfully imported into a new database. Tomorrow, I begin verifying the integrity of this reconstruction, and patch the few missing tables with backup data.

It’s strange… it’s not like we *didn’t* lose tables, but looking at the ones that managed to remain in memory for me to recover, it’s pretty much ALL of our important ones. The few player facing data tables that at this point are guaranteed to revert back a few months will be, I believe, hardly noticeable. Really low priority things like the decoration placements in fish tanks and messages sent in guild chat. I think there’s probably a technical reason for this, due to the REALLY important tables (creatures, players, packages, etc) being continually accessed, or larger, or something, but from my perspective it looks like a pretty cut and dry miracle.

That is, IF these data integrity checks end up going well. I’m hoping that will take around a day to do, but I have no plans to rush anything. If it needs to take longer, it will take longer. Once that’s done, then I’m firmly back in familiar territory: deploying the database to a production environment, hooking into it via our application and caching servers, and letting our code sing in harmony with our data once again.

Just in case you’re wondering, yes, I have located all of MY favorite pets in the recovered database 🙂 It’s been a bit lonely without my favorite water beetlie keeping me company, but he’s coming home soon.

<3 <3 <3

Update #7 From the Dev (May 5, 2026 2:30AM):

So. Much. Manual. Database. Construction.

And oh so worth it 🙂

The entirety of today was spent recreating a big chunk of our database using recovered files. We’ve got two hundred-ish different tables to recover, and for each one it goes a little like this:

In the fresh recovery database, an empty table of the same “shape” has to be created with the same name and settings as the table to be recovered
Specific settings need to be changed on that table to “detach” associated data
The recovered data file corresponding to the table is then copied in to the correct location on the virtual machine running the new database
A setting gets changed again to associate that recovered file with the newly created table and import the data where we want it
I perform a loose manual audit to see if the data looks and feels right (select * piped to a terminal that scrolls by really, really fast to get a general feel first, then a dig deeper depending on what the table is specifically)
Save the new state of the database to a persistent location
Start it all over again for the next recovered table file

I’ve finished this for a little over half of the recovered tables today, and… from my initial poking about they look *really* good. Like… I’m afraid to say “perfect” just yet, but I’m not NOT going to say perfect. I’ve saved the biggest, most important tables for tomorrow, and if this trend continues we could be looking at close to a best case scenario (all things considered).

Now, we’re not out of the woods just yet. Once I finish this reconstruction, I have to REALLY ensure we are able to trust this data. I’ll be spinning up an entirely separate machine and building another copy of our database from an older backup, and then checking for *any* missing sectors in the restored data compared to the verified good, but slightly older, data. If we’re able to surmise that the past 10 years of data is fully in-tact in the reconstruction without any errors, I’ll be comfortable moving forward with the assumption that the tiny percentage of more recent data is sound as well.

Sooooo, things are progressing, if a bit monotonously. This isn’t the kind of thing that I want to rush though, no matter how badly I want to have the game up and running for you all.

Finally, (and I may touch on this in a later post if I find the time and energy to write up some more info on the actual attack on our original database) I’d like to send a special thank you to the person who decided to maliciously wipe ALL of the files on the server. I believe that, in a very strange sense, they actually ended up *saving* our data from being caught by the ransomware that tried to encrypt the entire disk at the same time as the drive was being deleted. This is legitimately the first time I’ve ever been glad to have an entire disk deleted, because it’s the only reason I’ve been able to recover this data in-tact.

Pretty cool!

<3 <3

Update #6 From the Dev (May 4, 2026 4:30AM):

Well, as you might be able to tell from the timestamps that I’m slapping on these updates, my sleeping schedule has been twisted into a pretzel. BUT, I really think it might be paying off. I *still* can’t be sure of anything just yet (because until a fully functional database with good data is running in production it’s impossible to ever know that one little thing won’t tear down the entire house of cards), but tonight I’m going to get some sleep on a high note.

A lot of the past few days have been spent assessing, gathering, speculating, and building out a roadmap that can carry the rest of this endeavor to the target destination, Krafties running at full force, using the the *best* possible path to get us there. Ideally the perfect plan will keep us on that path as well. This has included doing things like

running multiple data forensic tools on the compromised disk to see what data might be available to us
checking older backups to ensure fallback options are viable for recovered data that may look good from the outside but end up being compromised
setting up new systems and environments to perform data extraction
setting up virtual machines/container systems to test recovery and dump recovered up-to-date data to more robust formats
creating a new upgraded machine to act as the final host for our shiny new database setup
completely rethinking backups (I’ll never trust anyone else with this ever again no matter how much they’re being paid)
ideating about how I might be able to make all of this excessive downtime for the playerbase more bearable while the database is under maintenance (since we still have ALL of our other servers up and able to communicate with the in-world game objects)
thinking about what special somethings I might be able to do to try to make this downtime up to players that are here, ready, and waiting
and maybe some other stuff too… It’s been a long three days.

Anyway, I mentioned I’m going to sleep on a high note. I’ve spent over a full day just experimenting with these recovered files to see if anything can be done with them. Database systems are complex, fickle beasts. Do you have *almost* every file, but happen to be missing someseeminglyunimportanttinyfile.info and thisfilelooksredundant.seeth? Well, attempting to spin up a new db without those files might just result in a mile long wall of alerts and errors all screaming about what a monster you are.

However, if you search the internet for long enough there’s usually a few magical nuggets of wisdom scattered about somewhere deep in an old forum. I searched long and hard, and I’m happy to say that as of a few minutes ago I managed to import one of the recovered data files, holding fully up-to-date data, into a new recovery database. I believe the entire thing populated without any data corruption! Querying it seems to be working perfectly, and… that makes me happy. This was only a very small table in the database that I was using as a test (since I have to more or less handle every table manually due to this silly workaround being used to get the software to stop screaming), but it does confirm that these recovered data files *might* be a viable path forward. I will be so indescribably happy if that’s the case.

I think that’s a good place to leave things for tonight.

I can’t express enough how much I appreciate you all for the patience and understanding that you’ve shown throughout this ordeal, and how bad I feel that it happened in the first place. Thanks for being awesome. We’ll get this sorted relatively soon (I hope), and Krafties will be stronger for it.

Update #5 From the Dev (May 3, 2026 1:00AM):

Today has been a roller coaster, to say the least. I’ve accumulated so much information in an incredibly short span, and it has somehow left me in a place that feels further away from knowing the best path forward than before. Every new development or piece of information opens up some small possible thing that we could do, but always seem to introduce a heaping helping of uncertainty around something else.

Hopefully some sleep will help. At this stage, it’s looking like it will be, in the absolute best case, *at least* multiple days from now before Krafties can be spun up with any sort of persistent data attached to it, and in all honesty I feel like it would take a miracle to hit even that target at the rate things are progressing. Without testing all of the possible solutions, I won’t know which will actually accomplish what needs to be done, and which will be best for the game moving forward. Basically, I have to keep trying stuff to figure out what works, and that’s going to take time.

After some sleep. Night night.

Update #4 From the Dev (May 2, 2026 6:30AM):

Recent versions of a few table files which were previously showing at a 0 size on the recovery drive have been potentially located in unallocated space on the corrupted drive. Some of these, if functional, will save us some headaches, and could prevent a good deal of manual admin work when the database goes live. While we’re not at a stage, yet, where it can be known if any of this data will work, I am cautiously optimistic-er.

Still no ETA, but it’s looking like things will soon be at the stage, as in later today, where it will make sense for a new system to be spun up to see how this recovered data plays within an actual database. Once that’s been played with for a while, it should give me some feeling about what a rough timeline for us might look like.

Update #3 From the Dev (May 2, 2026 1:00AM):

It’s been a long day, but the work seem like it just might be paying off. I don’t want to get my hopes up, or provide any misguided excitement for you all just yet, but I can verify that through the first bout of file recovery we have something that, from a very cursory inspection, at least looks like a good chunk of recovered data.

Files pertaining to tables (for those that aren’t familiar with database terminology, you can think of these like separate pools of information tied to separate categories of things that exist in the game) related to a lot of critical types of things are populating on the recovery disk with a non-zero size. That means that, at the very least, there’s *some* data there. How much? I can’t say at this point. Database storage is inherently complex. While I could attempt to infer from the method of data recovery being used alongside some of these file sizes looking good, I don’t think it would be wise to get our hopes up just yet.

Essentially, I’m going to continue working under the assumption that it’s possible that *nothing* at all can be recovered until the moment it’s all running and accessible from an operational database, because anything else would be setting us all up for even greater disappointment in the event that this data doesn’t work out for one reason or another.

With all that said, however, recent events have left me cautiously optimistic. Keep those fingers crossed!

Update #2 From the Dev (May 1, 2026 6:30PM):

Hey guys, another small update: a cPanel 0-day attack on our host hit us with a pretty intense ransomware attack. Unfortunately, cPanel was installed on the intermediary server that our regularly scheduled backups were written to as well, and it was compromised alongside our database server. What does this mean? Basically, if the filesystem recovery techniques that are currently being tested against the compromised system aren’t successful, we’ll have to backup to the most recent manual backup that I randomly decided to pull to an offsite location a few months ago. While I’m still holding on to a small shred of hope that we’ll be able to recover the more recent files using some recovery techniques, I want to keep you all updated with the progress and potential paths forward.

In the optimistic case, we’re able to recover all, or most, of our database data, move it to a new secured server, route things around correctly, and then we’ll be back in action (hopefully) around where everyone left off. Please, please cross your fingers for this outcome.

In the less optimistic case that we cannot recover the most recent database data, it would mean we’d launch a new secure server using the older database data, and all accounts, creatures, loot, etc would revert back in time. If this happens, I will do my best to make things right by players. While we wouldn’t have the fine-grained data about xp, inventories, purchases, trades, etc for the lost months in this scenario, there are quite a few real-time logs that feed through to my local machine while Krafties is running that tell me about things like specific items being dropped, player/guild activity, etc. From this information, I can extrapolate from the different log messages to see when players where active, and use this to do something special for everyone that was active during this time. This would allow me to, more or less, infer which players have been the most affected in order to provide individual remedies based on specific activity on a per-player basis, and potentially give some really special things out to compensate those most affected. While I’m not ready to start fully ideating on that just yet, if it comes down to it I’ll be looking to the community to see what you all would want to happen in that scenario.

BUT DON’T PANIC! I just wanted to be upfront and prepare everyone in case this is bad-bad. Work is still being done behind the scenes to see exactly what the next steps are that will need to be taken to get Krafties back in working order in the best state possible. Again, I apologize for the down-time and whatever downsides result from this issue. For better or worse, this is the first time that Krafties has ever dealt with something like this. In either scenario, it sure as heck is going to result in better, more robust systems and practices moving forward.

Annnd, of course all of this happened the day before a massive update was planned to hit for the NPC shop system, alongside some neat new items 🙁

There is still no ETA for when things will be back, but at the moment I’m doubting that we’ll be able to get something up and running by today.

Stay tuned! This page will be updated as new information becomes available.

Update #1 From the Dev (May 1, 2026 1:00PM):

Hey guys, checking in here to let you know that the server issues are being looked into. Of all previous issues, this one might be one of the worst… we’ll have to wait and see as things are investigated, but it’s looking like our main database server was attacked, quite possibly with ransomware. I’m pulling everything accessible to a safe location right now. Best case, it’ll be a matter of migrating recent live data to a new server. Worst case, we have to roll back to a database backup and then I’ll do a lot of manual work to make things right with players. Really, really sorry for the issues today. Things will be up as soon as possible, but I can’t say exactly when that will be at the moment.

Just in case anyone here runs any server that has cPanel installed on it, I’m sending this out to hopefully save you some heartache. This is the most likely culprit that got us: https://www.reddit.com/r/cpanel/comments/1syyajp/massive_cpanel_0day_auth_bypass_hits_web_hosting/

WORLD OF HUNGER EXPANSION

Known patched issues:

(8/5) Item images in Inventory are not showing for some players – FIXED: Item images should now display for all players (barring network issues)
(8/6) Transfer locked blueprints from the Bounty Credit board for Salt, Dairy Cream, Butter, Dough, Cheese, Tomato Sauce, Soup Stock, and Noodles cannot be crafted with – FIXED: Crafting recipes have been enabled
(8/6) Transfer locked blueprints from the Bounty Credit board for Salt, Dairy Cream, Butter, Dough, Cheese, Tomato Sauce, Soup Stock, and Noodles are giving transferrable results – INTENTIONAL(TEMPORARY): This is intended behavior for now, so as to keep crafting recipes simple and prevent the need for a bunch of different recipes for each locked ingredient type. The crafting system will be changed in the future to accommodate for this, and these crafts will be modified then. Anything crafted before that happens will remain transferrable.
(8/6) “Craft Item” button in Crafting app is slow to initialize – FIXED: The button should now respond within seconds of the app initializing
(8/7) Mining Rock objects become stuck in cooldown with a red outline – FIXED: Type “/123 fixrock” in local chat (while wearing your Krafties HUD and Tool Attachment) to acquire a magic rock fixing wand.
(8/8) Crafting experience awarded from temples is not boosted by food/event xp bonuses displayed in apps – FIXED: Crafting experience awarded now uses the new boosting mechanism
(8/9) Crafts are giving incorrect xp – FIXED: Crafting xp results have been corrected
(8/9) Oinklers have small visual artifacts on their backs introduced by SL lighting and sculptie weirdness – FIXED: Oinkler model data has been updated, with all Oinklers now having additional mohawk and tusk features