Topic on Simplicitypvp:General

Jump to navigation Jump to search

Region file cleanup

21
Yukar9 (talkcontribs)

We're going to be deleting some old regions that have not had any player activity. We're doing this to reduce the server's disk usage. Exactly when and what will be deleted has not been decided, but we will try to delete the minimum possible.

With some people flying extreme distances we've been seeing large disk usage increases, upwards of 300 GB per week. Thanks to those who have slowed down their traveling this increase has slowed down. But still this is simply not sustainable in the long run. So we're going to be deleting some regions that are part only of player trails and have not had any other player activity. We will be doing this cleanup before the update to 1.18. We are also considering on making it so that going forward chunks are not saved if a player only very briefly loads them, as a permanent solution to the disk space issue.

We understand that some people want to be able to follow trails to find bases, and that this will make it harder to do so. Some people believe that it's unfair that bases become vulnerable after every update, while some believe consider following paths to be a core feature of the game. Personally I'm sympathetic to both views. Keeping the status quo has been easiest so far, so that's what we've done until now. But this is simply not sustainable anymore, so something will have to change.

DKC (talkcontribs)

I think this is a great idea Yukarion. However, I think keeping this idea strictly to the overworld and end chunks will be the best option in the long run.

Many players, including myself, like to basehunt so allowing this fix to affect nether trails is a bit much. Considering that most players don't go past the overworld world border coords in the nether, I do not think taking one of the biggest and well known features, that being it's expansive nether full of old and historic bases, away from us as a player base is a good option. I understand that we need to fix our disk usage problem, but most of that is coming from massive overworld / end journeys. In my opinion, applying that to only the overworld and end would allow base hunters the freedom to basehunt and free Sim of its disk usage problem.

Although this change will affect basehunters negatively, because of overworld changes, I think this is possibly the best avenue for us to continue down. Sim is composed of many other types of players with varying degrees of playstyles, so it is unacceptable that I ask for no change at all, in order to benefit the minority. The proposed changes I have made however, will allow for basehunters and other Sim players to coexist. We can't have our cake and eat it too.

Yukar9 (talkcontribs)

I understand that, so we will prioritize end, then world, and then finally the nether only if we really have to.

To be clear, we're also not talking about deleting bases, while the exact number isn't decided, we're talking about only chunks that have been loaded for less than 30 seconds. We want to hit chunks loaded by elytra flying and not much else.

Yukar9 (talkcontribs)

One other option, is to not delete anything existing, but to only delete trails going forward. We could do a one-time storage increase if we have to, we just don't want to be in the situation of having to increase the disk space on a regular basis. What exactly we'll do hasn't been decided, and we do want to involve the community in this discussion.

KoriJenkins (talkcontribs)

The problem I see with the "delete nothing and delete trails going forward" option is that it basically heavily incentivizes people to abandon their current base and just move on.

Think of it this way. You have a trail of generated terrain to your current in-progress base, but flying 100k will get rid of ANY trail to you. Why wouldn't you just do that and be much safer? It basically gives newly built bases a huge advantage over extant ones.

If we were to target each dimension by update (doable by filtering beehives in the overworld, nether gold ore in the nether, and nothing in the end since it doesn't matter) we can prevent current bases from being disadvantaged and abandoned in large numbers. The only bases that I know of that predate update 1.14 and would have a trail leading to them still are Revalia and a couple of mine. Every other base if we deleted terrain in the overworld as old as 1.15 would be given that protection.

Though protection isn't the main goal obviously, if we're trying to eliminate side effects and impact all players as little as possible we wouldn't want to make them feel their base should be given up on because they could have a safer one.

Yukar9 (talkcontribs)

But that would leave us in the same situation as doing nothing. If we do nothing, bases will be vulnerable in 1.18 all the same.

KoriJenkins (talkcontribs)

I'm saying I think only deleting trails in the future will motivate people to abandon bases or make base growth stagnant as everyone would know whatever base they're currently at is vulnerable compared to a base they could make after the plugin is created.

We'd basically be saying "every existing base has a trail, but every new base going forward won't" which isn't really fair to existing bases. If that was our official policy it would lead to stagnation and abandonment of bases.

I'm not saying do nothing, I'm saying that shouldn't be our plan unless we plan to clean up already existing trails as well, not just trails in the future.

Couleur (talkcontribs)

Not sure if there is a way/if it is possible at all, but try comparing each chunk with how it generates naturally and delete it if it has the same blocks

That may take a much longer time but will probably trim down size by a lot more

4Billotforgotoldpassword2 (talkcontribs)

"Region file cleanup" Should always be a last option resort. I think what we need is more info on things such as cost for more hardware space. If you can, check how many gigs the server is increasing a month, and put that into a years time. Money is never sustainable, but it can be a for a few years until last chance resorts like this go into effect. So, if we get an estimate on how much we would need to pay for more server space that could last us a year or more. I think its appropriate to first release the numbers and possibly reevaluate from there/set up a donation page for that money.

If "Region file cleanup" was a necessary I think cleaning up end chunks that weren't loaded for more than a minute should be removed, but only in the end. Taking a guess the file size is 600GB we can probably knock it down by 80% and if that is not enough to keep the server from maxing out on space then should we really look to the nether then overworld. TLDR: Delete unneeded end chunks then reevaluate from there.

Keep in mind, if region file cleanups had to happen the order should be from end - nether then overworld. For overworld, only chunks that are post 1.14 and for nether, only chunks that are post 1.16.

Yukar9 (talkcontribs)

Yes, we'd definitely do the end first.

We have 6 TB disk now, and about 400 GB free. Disk usage was increasing by 300 GB/week, but is slowed down as we've been asking people to take a break from traveling. In the past 7 days, disk usage increased by 50 GB.

We can add an extra 6 TB disk for 18 euros/month, which I'm not opposed to. The problem is when people do start traveling again. That extra 6 TB might not even last us a year.

4Billotforgotoldpassword2 (talkcontribs)

When you say 18 euros/month for 6 TB, is it a monthly payment? So, in reality for all 6 TB it will cost 12 x 18 = 216 just for that 6 TB?

Yukar9 (talkcontribs)

Yes.

We can't add it to the existing server, so we have to order a new server and then move everything over. There's a fixed fee of about 100 euros for setting up a new custom server, there would have to be 1 month where we pay for both servers, and then there's the time to do the transfer. But yes, 18 euros per month.

Burger (talkcontribs)

Certainly clean up End chunks - following End trails is practically impossible anyway. I hope the Nether chunks remain indefinitely, as they are the only good non-holistic method of finding bases, which is an important part of Simpvp.

Question: once the cleaned-up chunks are regenerated (by new travellers), will there be a noticeable border with the chunks which were never loaded before? If so this move could ironically make basehunting even easier.

DKC (talkcontribs)

To answer you question yes and no, because the problem is loaded chunks not unloaded chunks when the creation of chunk trails are being discussed because what causes chunk borders is loading those chunks next to previously loaded chunks. So in the event they do delete trails, trails that were previously explored along the chunk border will have this problem, but trails that were not followed outside of their origin chunk will not. It'll be like nobody even passed through it if nobody went over the origin point of that trail. That being said, the move to delete trails created by elytras may also delete the trails of others loading outside of that origin point. It is really a toss up to say.

KoriJenkins (talkcontribs)

Something to consider regarding the 18 euros/month is that doing that would make a proper hardware upgrade over what we currently have less financially feasible.

The way I see it we have 3 real options with variables of them.

1. is to delete terrain outright based on age of the region file and time inhabited and create a plugin that does that automatically in the future.

2. is to only do the plugin (I'll explain why this is hard in a second)

3. is to upgrade the server and do 1 of the first two options when the need arises in the future.

I'll go on record saying I support a hybrid of the first two before I explain my position on why option 2 on its own won't work or option 3.

We're close enough to running out of space that there's a very realistic chance of us capping out on it even with a plugin running to slow the map's growth. It's why I think some level of option 1 needs to be included with it.

Option 3 + option 2 is a legitimate option as well as option 1 + 2 but as I mentioned at the beginning of the post, if we wanted to ever upgrade the actual hardware along with adding file space it would result in us probably not being able to do so. On top of that, we don't really have any donation system set up and there are complex tax laws to adhere to regarding doing something like that.

Realistically I think it's going to be 1+2 or 2+3. No single option on its own is a complete resolution to the issue.

4Billotforgotoldpassword2 (talkcontribs)

I'm gonna try to explain each dimension and its important role when it comes to simpvp and in order for which would be affected first.

Firstly the end. When it comes to end terrain there is no legitimate way to follow any sort of end trails, even if you found a few raided end cities. Following a line of raided end cities is near impossible if the raider changed up their pattern in flying i.e. flew in a straight line, (obvious pattern to follow,) but then suddenly turned and you've lost it. It would take extensive amounts of graphing or mapping (Xareos world map is gone and there is no way to map now.) Huge time waste and really not worth the effort. Also important to this and other dimensions; updates in terrain. There has been nothing said by Mojang about an end update that would make keeping end chunks important to keep in the sake of trail following. What can be said is chunks loaded for less than a minute, more or less, are completely useless and I think should be cleaned up.

Next is the nether. If any nether would to be deleted it should only be chunks loaded post 1.16. Following loaded lava in the nether is possible with a repeating pattern such as a straight point from A. to B. If the flyer constantly changed up their direction its nearly impossible and quite a large frustration. Once again, large amounts of documented graphing or mapping would be to be done to follow these trails, not to mention you yourself are loading lava too and in some cases you can confuse yourself in your own loaded lava if you're struggling to find where the trail turned. So, these post 1.16 nether chunks really serve no purpose, and it was just recently updated, so I don't see a update coming to it all in the near future.

Overworld is the most important when it comes to not deleting any chunks and if any were either post 1.14 or post 1.18 (will discuss more). Once the server updates to 1.18 following chunk trails should be very easy, but I'm not sure because of the way they made the border blend so well. Same idea with patterns, if the person flew in a straight pattern and you notice the biome blends the whole time, you got an obvious trail. If they sporadically flew, then it's just a little bit more frustrating like following trails from 1.7 to 1.12 and you trying to find where it picks up again. A very strong point that I think is important to all of this is we don't want to isolate bases making them too safe and giving others a disadvantage. Deleting trails to the bases basically gives them "tp'd" like conditions, as if just an admin did /tp'd to them. I think its important we keep that balance and make sure we all have equal advantage. That there is the main reason why I'm against deleting any overworld chunks for now. Now, 1.18, once the server updates it's likely many people will be loading new chunks, and with that will come larger server files. Along with extra space possibly saved by cleaning up the end, it can help us survive into 1.18 for a little while. Something that I would be for and could be done in the future is deleting any chunks made in 1.18 or in the future. Deleting 1.18 chunks really has no effect just like the end, it doesn't put anyone at a disadvantage, nor will it probably ever update again in the near future. (Basically suggesting regular basis clean ups in the future once the server updates)

KoriJenkins (talkcontribs)

End chunks and post 1.16 nether chunks are obvious. With the overworld it really is just a question of "do we think this terrain matters that much?"

What you say is true, there'd 100% be islands that look like they were TP'd to. Who would even see that though? There's no gigantic view of the map like what you imagine and never will be.

Lord1 (talkcontribs)

lame lame laaame, delete all unused chunks. no more base hunting for base hunting virgins. if you wana find someones base do it the old fashion way, espionage.

KoriJenkins (talkcontribs)

This might be the only time we've agreed on something. Though I don't think we should delete all unused chunks. I like seeing the age of terrain (think the difference between 1.13 oceans and 1.12 oceans)

Goujonsimpvp (talkcontribs)

just reset the map lol, we've been on it for so fkn long

Aaronen (talkcontribs)

BRO yukar this is awesome do it, remove paths that have only been visited by one way travelers.

i dont see the problem. new players wont suffer any negative changes. new terrain. new generation. nothing changes. everything is perfect. --araronen