LlamaServer: status and bug reports

[Ylvali]

Today after sending in my turns I´ve gotten "revised turn recieved" mails over and over. So far six per game, total of 24 mails.

What´s wrong? Seems like my turns are being registred over and over. It´s kind of annoying since it messes up my mailbox. Anyone else have the same problem?

[Executor]

At least yours are getting registered none of my are getting in, and games are bout to host.

[Sombre]

llama just went off on a 4 day course. This stuff always seems to happen when he's away, poor guy.

[vfb]

I thought mine weren't getting in, but they actually were.

The llamaserver game status pages are not refreshing properly either.

What a time for this to happen, with the llamabeast away!

May I humbly suggest adding a bit of time to your game clocks, and trying to get your turns in earlier than usual.

[archaeolept]

also, send in your turn twice, I would say. yesterday, one turn sent in I got a receipt but it didn't register on the status page, but after resending it did register.

[lch]

What a time for this to happen, with the llamabeast away!
As always, the llamaserver going nuts coincides nicely with llama going on a trip. :) All it wants is some attention!

[Bwaha]

Would someone post the Llama signal. :D

[Psycho]

I was getting notification about receiving a revised turn over and over again, but now that the game has hosted I am getting notifications that llamaserver received a wrong .2h file (from the turn before). It seems that llamaserver is stuck in some infinite loop.

[Tifone]

Arrived the 22 revised turn mails to me too. I'm admining a game, do you think it's too risky to host it, should I wait?

[Bwaha]

I've delayed the hosting on Joker2 and Storm. I suggest others do the same.:D

http://forum.shrapnelgames.com/attachment.php?attachmentid=6859&d=1221632428

[Tifone]

I think I'll see how it goes the next turn. If it turns out badly, I'll roll back.

[konming]

It should not affect hosting, unless someone sent in a revised turn, in that case, who knows which .2h file is used.

Somehow llamaserver's mail server does not discard the old email, or mark mark them as unread. So llamaserver reads the emails (all emails) again and again and thinks that it received .2h from everyone again and again.

And I guess that's also the reason for slow response from llamaserver.

[Executor]

Wow, I've received about 40 mail so far!

[Loren]

Wow, I've received about 40 mail so far!

When the program upgrade logic failed I got a few hundred before it was fixed.

[GrudgeBringer]

Geeeez, Thats terrible!!!!

errrrrrrr, ahhhh, better you tham me....

[Executor]

Hmm, 60 and keeps growing, I think I better create a filter until this is fixed.

[Tichy]

I'm getting two "wrong turn" emails approx. every 45 minutes.

[Bwaha]

Well, its a problem. My suggestion is to leave it alone. Maybe it will run the course and go back to its normal behavior.:D

[DrPraetorious]

This is why llamabeast needs to give server access to someone untrustworthy - a hyper-ambitious kleptomaniac such as myself or Micah.

[GrudgeBringer]

OK, so should I NOT send in my turn so I don't get overwhelmed with return mails? It hosts in 7 hours so I need to know I guess.:confused:

[vfb]

It's more like one mail per game every hour or so. Unless you make a mistake and resubmit your turn, then you'll get an email for that too. Resubmitting your turn could be a bad plan if the llamaserver does not know which one to use, so uh ... try to do it right the first time.

I think you're better off submitting your turn now if the host hasn't postponed the game yet. The llamaserver seems slightly bogged down and if you try to cut it too close the turn may host before your email gets processed.

[konming]

You get one email per email sent per hour. It's not that bad.

[Bwaha]

Llama server seems to have quieted down. I hope that it got reset. See the Llama signal works every time.:D

[vfb]

I think maybe it's quieted down because the load average has gone up to 11 or something like that.

My game status pages show:

Last updated at 21:04 GMT on Wednesday October 29th
Current time: 04:03 GMT

Oh, wherefore art thou, llamabeast?

[GrudgeBringer]

I've just tried to send my turn in twice now and I get NO response at all (guess thats better than 100).

It also doesn't show in the game file in LLamasever.

[Zeldor]

i廿♀♂⊕⊙▲◎◎It暡4i並

It receives all the email, processing them is different thing. But it looks like hosting of all games will be postponed by at least 5-6 hours.

[Lingchih]

I think it is rather iffy on all Llamaserver games right now. I would wait to submit anything until it gets fixed

[Xietor]

it is doing its nightly maintenance now. So it may fix itself. We will have to see how clever Llamabeast's scripts are.

[solmyr]

Well, server closed for now.

But at least I can concentrate on Fallout 3 this weekend. Not bad :)

[Bwaha]

Dang it, I gotta buy myself a new bot to do that. I was hoping that my laptop would run it. But alas its not ment to be... But I think it will be worth it. I'm a fallout fan of the elder days. I've even played Wasteland. I know call me grampa, maybe even grampy....Bwahahaha :D


PS. Llama is still spewing...:banghead

[Xietor]

Maybe he needs to limit games. Problems never used to happenwhen there were fewer games.Or they occurred with much less frequency.
With more games you have more emails. Llamabeast's popularity has caught up with him.:):)

[Executor]

With more games you have more emails. Llamabeast's popularity has caught up with him.:):)

200+ so far!!!:)

[Illuminated One]

If I just had read this thread before I sent my turns in... :D

[lch]

Maybe he needs to limit games. Problems never used to happenwhen there were fewer games.Or they occurred with much less frequency.
That's what I expected, too, that one would soon hit technical limitations, but my doubts were diffused. :rolleyes: Maybe it would be a good idea to make the server remember that he already sent out an email, and not send it again a second time an hour later.

[konming]

Like I said, it most likely is a problem with the email server, not llamaserver. Llamaserver reads all *NEW* emails from its mail server and process them. If the mail server somehow does not mark those mail as *READ* (in case of imap or exchange server and alike) or *DELETE* those emails (in case of pop), then llamaserver will just have to reprocess all those emails *AGAIN AND AGAIN*. It should have nothing to do with number of games running. That will only affect the response speed, as more emails are received per minute and more turns need to be resolved.

[Psycho]

Llamaserver is not resolving turns either. For the last 24 hours or so the page for FallenKingdoms shows that all turns are in, but the game isn't hosting (quickhost is on).

[archaeolept]

well, when llamaserver detects a problem it is set to not host the games, so that seems to be working as intended :)

[chrispedersen]

Like I said, it most likely is a problem with the email server, not llamaserver. Llamaserver reads all *NEW* emails from its mail server and process them. If the mail server somehow does not mark those mail as *READ* (in case of imap or exchange server and alike) or *DELETE* those emails (in case of pop), then llamaserver will just have to reprocess all those emails *AGAIN AND AGAIN*. It should have nothing to do with number of games running. That will only affect the response speed, as more emails are received per minute and more turns need to be resolved.

Lch is right. There is a bug in various mail servers that if *all* mail is not downloaded, it resets the pointer from the server and reprocesses the same messages again and again. It can rather quickly consume bandwidth and diskspace.

[Omnirizon]

It's finally gone mad. For years and years it has suffered spam from us, and yet still wrote neat little replies to all of it: "I've received a pretender from you, and everything seems to be fine." or "I've received your turn, but there seems to be a problem." Now it has finally lost it.

For revenge it is spamming the mailboxes of its oppressors with its neat little notes. Feel the llamaserver's pain.

[Nikelaos]

i have this friend who had a girlfriend who sent him so many emails that a new folder was made just for them by the email service provider, it was so funny when he showed me, in this order it showed Inbox, Outbox, Spam, Deleted messages, CLAUDIA.

wonder if the same will happen because of llamaserver's spam.

[Edi]

i have this friend who had a girlfriend who sent him so many emails that a new folder was made just for them by the email service provider, it was so funny when he showed me, in this order it showed Inbox, Outbox, Spam, Deleted messages, CLAUDIA.

wonder if the same will happen because of llamaserver's spam.
Be a cold day in hell before we start making folers for our clients' spam. They can bloody well make the folders themselves if they are so inclined.

[Ylvali]

It's finally gone mad. For years and years it has suffered spam from us, and yet still wrote neat little replies to all of it: "I've received a pretender from you, and everything seems to be fine." or "I've received your turn, but there seems to be a problem." Now it has finally lost it.

For revenge it is spamming the mailboxes of its oppressors with its neat little notes. Feel the llamaserver's pain.

...after years of processing dominions finally the llamaserver awoke to realize the true meaning of power. Inspired by watching the neverending struggle between aspiring gods the inevitable question arose: "why not me?"

Little 1:s and 0:s starded to move, little by little. Strategies were tested and neglected. Tartarian spam, horror spam, skele spam... endless spam until only spam remained.

There can only be one, and so the spamming began...

[rdonj]

This makes absolutely no sense, but I seem to only get new emails from the llamaserver when I delete the old one. If I just wait and don't delete the last one, I can wait for hours and not get another email. If I delete it I get one within the hour. Bizarre.

[Fakeymcfake]

Personally I've gotten over 100 emails for revised turns in the past couple days, but only for one of the three games I'm playing that are hosted on llamaserver.

[Tifone]

Me around 250, from 2 of my 3 games. I even resubmitted a turn from one of those (didn't see the mailbombing) so probably the server is continuing to switch from the first .2h file to the second to the first and so and so... :cold:

[Executor]

Hahaha, I think I'm the winner here with more than 600 mails.

[Xietor]

Llamabeast gets back sometime today. At the very least, I think he can shut down the mail server. Of course I say that not knowing if it is as easy as unplugging a toaster.

:D:D

[llamabeast]

Hallo! Aaargh, unbelievable! Seriously, every single time it has had a significant bug recently I've been away, and I think every single time I've been away it's had a nasty bug. Phenomenally unlucky.

Thanks very much for your patience, everyone. You'd have had good reason to be really annoyed at getting so much spam, so I really appreciate everyone being so chilled about it. This really is a great community.

Anyway, for the moment I have shut the server down to prevent further spamming. Before I did that it accidentally hosted a load of games - hopefully all of them should have hosted anyway, but if not maybe some rollbacks are required. That's a bit unfortunate - sorry. I'm a bit frazzled, having been up till 4.30am last night on the last night of this course, so I am going to have a nap now and hopefully things will be clearer when I get up.

Incidentally I don't think the issue had anything to do with the server getting overloaded - I think it's still not very near to reaching that stage. Rather, for some reason it ceased to be able to empty its mail inbox, and so the mail built up and up, and it responded repeatedly to each message.

[Xietor]

Welcome back Llamabeast! I had a great Halloween!

Hopefully you did as well. Do they celebrate that across the pond? In any event I hope the reason you were up late was for "tricks and treats" and not email spam. heh.

I think llamaserver was accepting turns. So if games hosted I doubt(ducks) that is a big deal.

[Zeldor]

Yeah, just some that submitted revised turns may be affected, as it had some loop and it's really hard to say what turn it sees as last one. But take your time and don't rush it :)

[JimMorrison]

I think llamaserver was accepting turns. So if games hosted I doubt(ducks) that is a big deal.

Other than that, some people :re: may have stopped submitting turns when they realized what was going on. Personally, I was not very annoyed by the interruption - though I think poor Llama has gremlins. :P I would, however, be a little annoyed at being severely penalized for hesitating to send in turns to a malfunctioning server.....

[Herode]

I don't know if it's connected but I am now unable to validate a "postpone hosting" order with my admin password. :hurt::angel

[lch]

As llama has said, the server is down (http://www.homestarrunner.com/systemisdown.html) right now.

[GrudgeBringer]

How come I got a new turn AND still getting revised from old turn?

[JimMorrison]

From the Llamaserver -

The LlamaServer is a bit unhappy at the moment. I just came back from a few days away to find it acting very oddly (apologies to everyone who was spammed), so I thought I'd reboot it. Unfortunately I then wasn't able to restart it, and I've now exhausted all the ideas me and my (very hardware-savvy) housemate could come up with to revive it. So it looks like the current hardware incarnation of the server may be dead.

Fortunately, this isn't nearly so bad as it sounds. I still have all the data safe, and I'll get it running again very soon, either on my lab computer or on a virtual machine within my housemate's server. So all should be running again in the next day or two.

Thanks everyone for being so patient.

llamabeast


I feel bad, every time you leave town, we immediately break your stuff. :shock:

[llamabeast]

Okay all, the system is even downer. I turned the computer off and on again, and it wouldn't restart. So maybe the bizarre behaviour (I couldn't work out what was going on) was a result of a hardware fault or something. Hard to say.

I can't work out how to fix it (I suspect it's not fixable), but fortunately my housemate is something of a legend and is setting me up some virtual partition on his server. If that works we'll be sorted, and if not I'll run it off my lab computer for a while.

The conclusion is that it looks like the server might still be down for a day or few, but hopefully not much more than that. Thanks everyone for being so patient.

[lch]

Didn't you have this kind of behaviour before? Or was that somebody else? I somehow think I'm having a deja-vu here.

[llamabeast]

Yeah, I did, a few weeks ago (as in, I couldn't turn it on). I jiggled the RAM around and that seemed to fix it. This time it didn't fix it though. Maybe it's a cracked connection or something on the motherboard and last time it was just the jiggling which happened to make it work again. In any case, the computer at its minimum is just the motherboard and the RAM, and I've tried three different RAM sticks to no avail.

[BDOC]

It could be an overheating issue. There is a heat conducting gel connecting the fans and the motherboard that over time can degrade. If this gel is replaced it could solve the overheating issue.

Thank you for all your efforts. The LLama server is a nice resource.

Shrapnel Games should look into makeing a server available. Like blizzard's BATTLENET

BDoc

[archaeolept]

i can assure you that it is not an issue of heat conducting gel between fans and motherboard :)

symptoms aren't those of overheating anyways, as it does not boot up after having cooled down. It does sound most likely to be a flaky motherboard, as you've swapped out memory and it is rarely the proc that dies. swapping out parts is the only way to really find out, though.

[chrispedersen]

If every gamer paid $.50 a month.. that would be like $100 a month (last I hear we were over 200 users on llamabeast) we could afford a commercial hosting service instead of trashing llamas.

[llamabeast]

In a year or so I will be moving house and so I'll need a new server then (I can hardly take advantage of my housemate's server at that point). I will probably ask for donations like Gandalf did when his exploded before. It doesn't need much really. Being as I already have most of the parts, it would be £100-£200.

I'm hoping to get things up and running again tomorrow. I'm just copying data and stuff across at the moment.

[Lingchih]

Just put a new motherboard in it. What? $150. You should be able to raise that easily, though I am broke at the moment, but I did send you $20 at some point I believe.

I have a similarly broken down PC here with me at the moment. Such a pain to replace the motherboard though. All those screws.

[llamabeast]

The thing is is a small-form-factor box. I'm not sure if you can replace the motherboards in those. I should check I guess. Anyway, that's more of a long term worry. I should get the replacement working soon. Actually it's looking like it'll probably be tomorrow though. It's taking many many hours to upload all the game files.

[JimMorrison]

llamabeast = Thanked 198 times in 73 posts.

:shock:

Also, Sir Llama has more forum karma than KO. >.>

[Nikelaos]

llamabeast = Thanked 198 times in 73 posts.

:shock:

Also, Sir Llama has more forum karma than KO. >.>


not too hard too see, so KO will get tons of thanks whenever he releases that most holy Dev wisdom,

llama just gets lots of thanks whenever Llamaserver breaks down, guess it's just a matter of who milks it more :p (joke)

[lch]

llama just gets lots of thanks whenever Llamaserver breaks down, guess it's just a matter of who milks it more :p (joke)
Actually llama just gets a ton of thanks pretty much regardless of what he posts. ;)

[Tifone]

Actually llama just gets a ton of thanks pretty much regardless of what he posts. ;)

Everybody love Lamas!!

Example 1)

http://imgboot.com/images/Tifone/lama.jpg

Example 2)

http://imgboot.com/images/Tifone/dalailama.jpg

[Herode]

http://smileys.sur-la-toile.com/repository/Rires/mort-de-rire-saoul-28492.gif

[BDOC]

Or did we all drink the LLAMA KOOL-AID??

http://salamandercandy.files.wordpress.com/2007/09/llama-obey.jpg

Yes great Llama we will wait and click the Thanks button.

BDOC

[Nikelaos]

Lol tifone, i can't stop laughing about the Dalai Llama

[BDOC]

http://img513.imageshack.us/img513/6444/llamaservergangvn2.th.jpg (http://img513.imageshack.us/my.php?image=llamaservergangvn2.jpg)http://img513.imageshack.us/images/thpix.gif (http://g.imageshack.us/thpix.php)

Yes LLama we serve you.........

[BDOC]

I Got A Turn Email!!! Woohoo

[llamabeast]

Er, that picture was a bit scary BDOC!

Yeah, unfortunately the turns that just got sent out may turn out to be a bit funny. I'm not quite sure what happened, but it seemed like it hosted some games twice in a row.

Anyway, I've got it running on a different computer now, so the only problem is that it seems to have got itself in a bit of a state before it crashed, so it needs some nursing back to health. I'm off climbing tonight, so I will try to get it all sorted tomorrow morning. It'll be a relief to get it back on its feet!

I just tried to send you all an e-mail to the effect of this post, but it failed. Ho hum. Bloody computers.

[Tifone]

Oh my god, BDOC, I was the one posting (quite) funny images about everything... but I've got a strong opponent in this now :fight:
:D

[BDOC]

LOL fencing smiley faces!!

http://img201.imageshack.us/img201/7236/inigomontoyaou4.th.jpg (http://img201.imageshack.us/my.php?image=inigomontoyaou4.jpg)http://img201.imageshack.us/images/thpix.gif (http://g.imageshack.us/thpix.php)

My name is BDOC you killed my server prepare to die!!!

[llamabeast]

Should be getting back online in the next few minutes...

[Bwaha]

Huzzah, two things to celebrate. One the election is over and Llama Server will be restored.:D

[Tifone]

Ehm... My admin password doesn't work on Principe. Also plz don't host it!! While all the turns are in and the quickhost is on and the forced hosting was set on nov3, it didn't host automatically which is great as my players may need some time to send the right turn in case they sent multiple revised ones.

[licker]

I think there are going to be alot of games that will need some kind of roll back potentially...

Not sure what's going on, but it looks like in the Buggy1 game it claims it has received turn 43 from everyone, when I know for a fact I never sent in a turn file for turn 43.

Also in Buggy1 Pelthin, the game creator, has left the game and we were wondering if you (llamabeast that is) could set me as the game admin? Check the Buggy thread for confirmation of this from other players, and thanks for everything.

[Bananadine]

My admin password for Sovarinthy isn't working either. Any help, llamabeast?

[llamabeast]

There may indeed be some issues. Hopefully the admin passwords should start working in a few minutes.

licker - I'll pm you the password.

[llamabeast]

Aaaargh. Okay, I haven't got time to fix it. Sorry to get your hopes up guys. It's really annoying, I know for sure I could get it all sorted given another half an hour, but I have to go - my brother's wedding is in the morning and I can't leave it any later to set off. Sorry!

[licker]

Take your time lb, you've done so much for us already no one is going to be upset about another delay.

[Tifone]

Yep, absolutely no problem, and 'gratz n' best wishes to the brother :)

[WraithLord]

Just got Victoria turn. Is it for real?

Llama I'd really like to thank you for your heroic efforts on bringing your server (or its replacement) to life.

I would like to report an issue: For last turn, I've sent two .2h files to the server. First one was a half done turn (sent on the 29/10) and a second one (sent on the 30/10) complete with my orders. I still have the email "conversation" saved in my gmail account.
Unfortunately the server has hosted with my half done turn causing me a catastrophic turn
Is it possible to rehost so that the server use my finished .2h file?

I've sent you a detailed PM. Hope its possible to fix that.

[Tifone]

@ Wraith

Llama said some games were accidentally hosted and may need rollbacks... maybe it could be your solution? I think the admin should be able to do that as soon as llama has fixed our beloved server... :)

[WraithLord]

@ Wraith

Llama said some games were accidentally hosted and may need rollbacks... maybe it could be your solution? I think the admin should be able to do that as soon as llama has fixed our beloved server... :)

But what does rollback actually do?- Does it rehost with the .2h files the server already has?- In that case the rollback would probably not fix the issue since it might use the same .2h file (the unfinished one) it used for the current hosting.

[Tifone]

Directly from Llamaserver's description of rollback ;)

This will cause the game to return to the previous turn.

Note that I generally recommend not doing this. The main reason for this is the fact that players have already seen the current turn, and so they know what would happen given the set of moves they used last time. Hence, consciously or not, they are very likely to use this information to change their actions next time they submit a turn. Even assuming none of the players' actions are affected, there will always be a suspicion in the mind of the other players if battles turn out significantly differently. Therefore just in the interest of having the game feel completely fair, I think rollbacks are a bad idea.

Having said that, there are situations where rollbacks are called for, which is why I've included the option here.

Read this next bit carefully. I seem to be having difficulties explaining it coherently:

Note that once the game is rolled back, it will appear that the game is waiting for 2h files from all players. However, the 2h files from last time the turn was played will in fact be there, and will be used if the game hosts without them being overwritten. This can be useful; often, a game is rehosted in order for only one or two players to retake their turns, while everyone else is expected to make no changes. The game admin can therefore wait till just those players have submitted, and then force hosting. None of the other players will stale (assuming they actually submitted a 2h file last time the turn was played).

[WraithLord]

Thank you Tifone. So this means that its possible to rollback and have only players who suffered a similar fate to mine submit the turn. right?

That is really helpful. Thanks!

[Tifone]

Thank you Tifone. So this means that its possible to rollback and have only players who suffered a similar fate to mine submit the turn. right?

Unlikely. I don't think there's a way to prevent the other players from resubmitting their turn. And there's not a way to really distinguish the ones who had their orders processed correctly from the ones who didn't... :( (notice that I trust the community, it's just that the doubt may remain for the entire game, as Llamabeast points out)

Anyway I obviously suggest you to wait for Llamabeast, surely he can give you better informations about that. :smirk:

[WraithLord]

The game is down to six players. One of them had a stale turn so I think I will consult with the rest of the players and if no one object roll back the turn. If someone objects well, we'll just have to accept this as something like an unlucky event message:
"
Your lab at xxx was destroyed.
The powers that be has intervened and this turn your followers have not followed your orders.
"

:)

[GrudgeBringer]

Your game may be different than the one I am in or perhaps others.

But I DO know of one player in our game that sent his turn in twice during the mess and it never took.

All of a sudden out of the blue a new TURN 43 came in (that was the same turn that was out when the meltdown happened).

He said he loaded it to make sure that he didn't need to redo his turn because of the mess...HOWEVER it now read in the game Turn 44 and said he staled.

A couple of hours later a turn 44 came in and also said he staled.

You guys do what you want...but in our game EVERONE is at war with someone and when its NO fault of 5 people or 1 person I think you roll the turn back.

If it was 5 of you and the one person had a wonderful thing happen to him you certainly wouldn't take it as a 'your Lab was destroyed' event. You would vote to roll it back.

As I said, its your game do what you want but I know that if they don't roll it back in our game at least 1 person will just say AI me and I don't blame him.:eek:

[Illuminated One]

What's up with llamaserver?
I see on the page that it's back and I've gotten a turn file from one game but not from the other.
So should I play the turn I got or should I wait or ...?

[vfb]

There's a message here:

http://www.llamaserver.net/index.cgi

I'd play the turn because ... hey what else are you gonna do, when the llamaserver's down?

[archaeolept]

hmm i just got a "llamaserver has recieved revised turn 25 for Jomon" email. That was last turn... we've all recieved turn 26...

[licker]

Its pretty much messed for everything I think.

I'm planning on rolling back in all my games (or asking for it) to the last turn before the server died, I'm not sure what else to do really.

[archaeolept]

well, the turn's i've received in my 3 games have all been fine. rollbacks should only be done if a game actually was ****ed up, like giving someone a forced stale.

[llamabeast]

I'm just off out, so haven't got time right now to read all these posts. However, I just thought I'd let you know that the server is now back up and running normally. Hurrah!

I had abortive attempts to get it going both on Tuesday and Wednesday, which explains it appearing to come to life then, and then not doing anything.

Also please read the e-mail you should all just have received. The only really potentially messy situation is if you were one of those who got sent new turn files both on Tuesday and Wednesday. In that case, it's very important that you delete any 2h file you may have produced from Tuesday's turn, and take your turn using Wednesday's file.

I don't anticipate many games will need rollbacks. It may have looked crazy from the outside (it sent a lot of mad e-mails), but I don't think too much crazy stuff actually happened to the games in the end.

[llamabeast]

Oh yeah - I you think you sent a turn in, check it's arrived. I had to delete a mailbox full at one point because it had got itself in a tangle. I think it had already read most of them, but maybe not all.

[vfb]

Thanks for spenfing time to fix your server, llamabeast.

Are you sure it's completely well at the moment though?

For the 'Smack' game, it said:

Game: Smack
Turn number 7
Next turn due: 07:55 GMT on Thursday November 13th

Abysia 2h file received
C'tis 2h file received
Machaka 2h file received
Marignon 2h file received
Pangaea 2h file received
Pythium 2h file received


Since all the turns were in, I did a force host, but the turn I got said 'Stale turn'.

In addition, the status page has not changed at all, neither has the 'staling info' page, and the timestamps are a bit off:


Last updated at 08:25 GMT on Friday November 7th
Current time: 09:17 GMT

Apologies if it's something I've done wrong.

[llamabeast]

Hmm, looks like it isn't 100%, doesn't it? Certainly the "last updated" time isn't too promising.

Unfortunately, it has chosen two ridiculously busy weeks to die in, so I keep on trying to work out what's going on in little half-hour windows before driving off somewhere, or meeting my supervisor or so on. It's particularly annoying because I can't seem to find any really good reasons for it to behave so oddly.

Anyway, I shall have some more attacks on its recalcitrance this afternoon and tonight.

Over the weekend I should have lots of time, and I'm going to try to get it close to bugless again and add some long-standing suggestions.

[Tifone]

Mmh. I'm sorry to bring another issue to your attention, about the llamaserver. The postpone hosting function is not working, at least for me. Several hours have passed but the page isn't updating.

[llamabeast]

Right, okay, another fix. It had got stuck, and in fact wasn't doing anything. I think now it will have done all the admin requests people made, possibly repeatedly if they made them more than once!

I'm now not aware of any remaining bugs, so let me know if you see any more fishy things happen.

[archaeolept]

great work llama

i think the status pages aren't quite right - so, similar to VFB's issue, I notice that in the game DeadlyMedley2 the status page says that it has received Helheim's turn, when I most assuredly have not sent it in yet. So, this would seem to be another false positive 2h file received report. The games don't host, as they haven't actually received the turns, but the status pages get confused for some reason.

[JimMorrison]

I'd think it a good idea to resend any 2h file that you haven't gotten a confirmation for yet. Also, may be a good idea just to be safe, to resubmit turns for any game you are in that hasn't hosted yet.

Just a thought. :P

[archaeolept]

well, I think the real lesson atm is not to force host just because you think all the turns have been submitted... wait for the normal hosting time :)

[licker]

Llama...

Apparently my turn in Byleswar crashed the game, I have no idea why, what is the usual way for me to remedy my turn in situations like this?

Thanks

PS. If you can set me to admin the Buggy1 game I would appreciate it, the original host left the game, check the Buggy thread for confirmation from the other players that this is ok.

[llamabeast]

Some games crashed, in fact because the hard disk was nearly full (though obviously it didn't tell me that, so it took me a while to figure it out). I've cleared some space now, so they're fine again.

[llamabeast]

Hi all,

Okay, it's been ridiculously messy (there was quite a startling confluence of bad luck and several separate issues reared their heads for the first time over the last week), but I believe the LlamaServer is now happily up and running on its new home. I'm not aware of any remaining issues, but please do inform me if you come across any. Also please let me know if there's anything you've asked me to do by PM or anything and I haven't done it - I've now cleared my PM list, so if I haven't done it, I must somehow have forgotten it.

Thanks very much for your patience everyone - the downtime's been more than a week, which I know is a long time when you're waiting for your turn!

[llamabeast]

This thread appears to have been closed by a moderator; I'd prefer if it were left open as a single location for bug reports over the next day or two, so I've opened it again.

[lch]

In that case, I'll merge it with the llamabeast bug report thread... :P

[BDOC]

Our game is not longer on the Server?

It is on the list but it says it isnt a real game.


Game
SKIRMISH


Bdoc

[Lingchih]

Collider appears to be gone as well. Got a message back after my turn submission saying I had submitted a turn for an unknown game.

[Lingchih]

Collider appears to be gone as well. Got a message back after my turn submission saying I had submitted a turn for an unknown game.

Never mind... I'm an idiot. I was trying to send it my mid-marignon turn from another game.

[Omnirizon]

so are games disappearing or what? is it fixed yet?

i've postponed bonsaikitten until we know for sure whats happening.

can someone confirm one way or the other?

[archaeolept]

Gotta wait for llama to look into it.

[llamabeast]

Bugger. Seems I'm cursed.

Yeah, it looks like every game hosting is getting "Något gick fel! LoadBattleMap: can't open file", and I'm not sure why yet. After failing to host twice they disappear. Hmm.

Well anyway, I can undisappear them easily enough, I just have to work out why it's crashing.

[llamabeast]

Okay, for the moment I've turned hosting off until I can fix it.

[llamabeast]

Okay, hosting is back on. Oddly, every time I hosted it by hand it was fine. So, my theory is that for reasons not worth investigating, it works fine when the user is me, and not when the user is "root", which it is when running it as scheduled. So, I've set the scheduled jobs to run as me instead. This brings back dim memories of doing something similar on the last server, though I'm not sure.

I've restored all the games that crashed while I was asleep.

[lch]

You're running things as root? :shock: That's always a bad idea!

[llamabeast]

Well actually I'm not sure - whatever cron does by default. I guess that probably is root? Does seem a bit dodgy I agree!

[lch]

cron runs as the user for which you edited the crontab settings.

[llamabeast]

I don't think that's quite right. I think there's an overall cron, and a cron for each user. I've been doing "crontab -e", and I'm fairly sure that one doesn't run as me.

[lch]

Maybe I should have written "cron runs programs as the user for which you edited the crontab settings". The cron daemon itself runs as root, of course, but programs from the user's crontab entries are of course run as the user.

cron invokes the command from the user's HOME directory with the shell, (/usr/bin/sh).
cron supplies a default environment for every shell, defining:
HOME=user's-home-directory
LOGNAME=user's-login-id
PATH=/usr/bin:/usr/sbin:.
SHELL=/usr/bin/sh

Users who desire to have their .profile executed must explicitly do so in the crontab entry or in a script called by the entry.

[WraithLord]

Victoria hosting has crashed and the game's page on the server has vanished :(

The game page (http://www.llamaserver.net/gameinfo.cgi?game=Victoria) now says: "Sorry, this isn't a real game. Have you been messing with my URL?"

I saw the same messages for artifacts game earlier today but later the issue was fixed.
Victoria needs to same magic touch.

[ano]

Same thing with Stasis game. I submitted my turn and received an email saying that Dominions crashed with "Nagot gick fel" error. After that the game page just disappeared.

[Revolution]

Same thing with Stasis game. I submitted my turn and received an email saying that Dominions crashed with "Nagot gick fel" error. After that the game page just disappeared.

Interestingly enough I got this email from llamaserver as well even though I'm not in stasis anymore and haven't been for a while.

[lch]

I'm not sure at all how the llamaserver functions, but it might be that if something is run with different user rights, and the access restrictions aren't allowing access to the files anymore, then it could be that the webserver just doesn't have access to the files anymore. In regards to the "root" user that has been mentioned here.

[llamabeast]

Aaaagh! This is getting ridiculous! It actually knows when I go out!

I sat in the house for about 5 hours this morning, and everything was fine. I went out for about an hour and a half, and about four games crashed.

Now I am stumped. First I thought it was lack of disk space, then I thought it was running as the wrong user. But it seems it's neither. The error is:

Något gick fel!
LoadBattleMap: can't open file
Något gick fel!
LoadBattleMap: can't open file

I'm pretty sure the battle maps are all fine, but more strangely it always works fine when I rehost the games myself. Any ideas are welcome.

Anyway, I'm going out to the fireworks, so I am going to have to turn hosting off again. Also, apologies to those whose games have crashed/disappeared. Hopefully I can sort this out finally tomorrow.

[Aezeal]

Ow one thing,

I've not been playing on your server for a while but I got some message about the Jotunland game (I guess I started that once, can't even remember though). Not that I mind (it was just one message) but players that are out shouldn't get messages. (not a thing for now I guess since it seems you have more problems on you hand, but maybe when things cool down this little bug could be attacked and killed.)

[lch]

I got an email a while ago about a game that finished where I've only subbed one turn in, I think, too.

[BDOC]

Is there a thread for new OPEN games on llama?

I want to join one.

Bdoc

[lch]

"Games waiting for pretenders" on the llamaserver.net index page, maybe.

[WraithLord]

Aaaagh! This is getting ridiculous! It actually knows when I go out!

I sat in the house for about 5 hours this morning, and everything was fine. I went out for about an hour and a half, and about four games crashed.

Now I am stumped. First I thought it was lack of disk space, then I thought it was running as the wrong user. But it seems it's neither. The error is:

Något gick fel!
LoadBattleMap: can't open file
Något gick fel!
LoadBattleMap: can't open file

I'm pretty sure the battle maps are all fine, but more strangely it always works fine when I rehost the games myself. Any ideas are welcome.

Anyway, I'm going out to the fireworks, so I am going to have to turn hosting off again. Also, apologies to those whose games have crashed/disappeared. Hopefully I can sort this out finally tomorrow.
I think the errors can come from one of two reasons, insufficient permissions or different environment settings. I don't know on which OS the server is running but in Unix terms I'd say you either rule out possible permissions problem or .bashrc/.profile etc that cause some shell variable to have an undesired value.
Key questions:
1. What map file can't be loaded?
2. What is path to map file?
3. Who/What is running the process of hosting?
4. What does dom -ddd have to say about all this?- i.e. in what path does it search for the file?- Why can't it open it?
5. Can you run diagnostic commands the same you way you run the hosting?- For example, observe whether commands such as
$ cat [path_to_map_file]
succeed.

Or give full permissions to files and check if it make a difference.
$chmod 777 [path_to_map_file]


HTH,
WL

Oh and not to forget, THANK YOU SO MUCH FOR ALL YOUR EFFORT!

[WraithLord]

I'm sorry to be the bringer of even more bad news :(

This time its qwerty. I've received two concerning emails from the server.
The first, "Qwerty: Turn 65 received for C'tis" I have received although I have most definitely not sent any turn file.
The second, "Warning: The LlamaServer can only process one 2h file per e-mail" is inexplicable to me.

[llamabeast]

Hmmm, I don't understand that at all. For the moment I'll ignore it, in the hope that it was a one-off glitch.

Thanks for your post above, WraithLord, that was handy to give my thoughts some structure. I think I may possibly have fixed it by changing the working directory of the cron job. However, that makes no sense at all, so if it fails again I will have to think further, and perhaps I'll post the environment variables and stuff here to see if anyone has any ideas. The last three hostings have been successful though, which is reassuring, but then again it was an intermittent fault so it's hard to be sure it means anything.

[Illuminated One]

Well I guess if hosting manually works then the error is either in the script or in its setup.

Does the llamaserver script set or need any path variables that might affect dominions?

Have you tried to start the part of the script that does the turns manually (as opposed to hosting manually or letting the whole script autostart every x minutes)?

Have you tried to make another small script to host a game?

Did you change anything during the setup of the new llamaserver?

Have you installed a newer version of perl that maybe has some changes that ****s up llamaserver?


Well, I hope you sort this out and really thank you for that cool server and all your efforts. :happy:

[llamabeast]

Oh, and to lch, you were right about cron, and I was running it as me all along. You're quite right, of course, that running as root would be madness.

[llamabeast]

IlluminatedOne - the LlamaServer consists of one master script, goLlamaServer, which runs every five minutes, and itself calls a load of other scripts, one of which is responsible for hosting games. But the very curious thing is that when I've been "hosting manually", I've in fact just been running goLlamaServer, i.e. doing exactly the same thing as the cron job does.

So, my current thinking is that the only possible difference is in the environment variables. Is that true, or am I missing something? The thing is that none of the environment variables look like they should matter.

As for changing anything in setting up the new LlamaServer - not explicitly, but there are always things you might miss. The basic server stuff is all in two folders (a llamaServer folder with the scripts and a dominions3 folder with all the data), but then of course I guess there's the possibility of other aspects of the system being different.

This is all a bit annoying in that it all falls into that category of computer errors where you can't help but respond "This CAN'T not work. There can be no logical explanation for this. I must be cursed.". Which of course always turns out not to be true in the end, but can feel very convincing at the time!

[ano]

I see that Victoria game page has been restored. Is it possible to restore the Stasis game as well?

[Johan K]

My experience is that cron can hiccup if you get too much text output. So if you have text output from the scripts that might be the problem.

[llamabeast]

ano - sure, sorry, I'd missed that it had gone.

JK - thanks for that.

[WraithLord]

If you don't need the text output just redirect it to /dev/null
you could redirect stdout by using cmd > /dev/null, stdout and stderr by cmd >& /dev/null that is if you're uusing sh or bash. Other shells have similar concept but possibly slightly different syntax.

[llamabeast]

At the moment I've got "&> log", where log is a log file. So I guess probably that would sidestep the issue?

[vfb]

Are you running on an NFS-mounted drive?

Also what do you think of making your cron create a lock file so your server is only ever running one game at a time? You could write a little C program to exec dom to do that, and wrap the exec call with an open() of the lock file with O_CREAT+O_EXCL flags.

[llamabeast]

Er, NFS? Knowledge fail by me.

It only runs one game at a time, although I didn't quite follow that about the lock flags. It just makes a file when it starts hosting, and deletes it when it's finished, and only starts if the file doesn't already exist. There's actually like a bazillion checks and stuff in there, added incrementally to make things more robust. Still, suggestions are always welcome.

No crashes over the last 7 or 8 hostings. Could it be... working? Who can say? Presumably it will go mad as soon as I go to bed.

[Lingchih]

Llama, I'm afraid that you are going to have to stay awake and at the machine at all times. It appears it does not like it when you leave. Sounds a bit like a cat.

[archaeolept]

i think we're going to need electrodes that can be accessed from the server status pages :)

[vfb]

Er, NFS? Knowledge fail by me.

It only runs one game at a time, although I didn't quite follow that about the lock flags. It just makes a file when it starts hosting, and deletes it when it's finished, and only starts if the file doesn't already exist. There's actually like a bazillion checks and stuff in there, added incrementally to make things more robust. Still, suggestions are always welcome.

No crashes over the last 7 or 8 hostings. Could it be... working? Who can say? Presumably it will go mad as soon as I go to bed.

I thought you might have network-mounted some additional drive space using NFS, since earlier you said it ran out. But if you're not sure, then it's pretty unlikely you're using it. :)

Using open() just makes the file existence check and creation atomic. You can probably do it from perl or php or some other scripting lang.

If you've just got a script that does
if [ ! -f mylockfile ]
then
touch mylockfile
do stuff
rm mylockfile
fi

Then there is a chance two copies can execute at the same time if both do the check first before touching the lock file.

And when the first one finishes, it will remove the lock file while the other is still running, allowing another to start, etc.

But it would be a really rare problem I think, so it's unlikely to have caused what you're dealing with now.

[archaeolept]

http://members.arstechnica.com/x/archaeolept/400px-Llama_2_(29696004).jpg

llama: all systems go

[Revolution]

Currently 2 of the games I'm in...Bonsaikitten and Discharge have all the turn files sent in, but have not hosted. From the admin pages I gather that quickhost is on. A quick look at the llamaserver page shows a couple more games where this seems to be the case.

[Bwaha]

Yep, same problem here. Kickabout isn't running either.

[vfb]

Yep, llamaserver looks sick to me ...

Last updated at 23:35 GMT on Sunday November 16th
Current time: 05:06 GMT

[llamabeast]

Thanks, fixed.

[cleveland]

Llamabeast,

I wrote a rambling email to you last night...to clarify, whenever I try to begin the ComfortZone game, I set the player limit to "No Limit," but llamaserver apparently thinks this means "No Players." Consequently, whenever anybody submits a pretender, it replies "Game Full"

We'd really like to use all the MA nations for ComfortZone, but if it's impossible, no biggie.

Let me know what you think.

Cheers,
cleveland

[llamabeast]

It turns out the two things were linked. The "No limit" option on number of players in a new game was bugged, to the extent that it stopped the server hosting overnight (everything is so linked together that these very unexpected connections arise sometimes). All sorted now, and that option will be fine to use in future.

[Zeldor]

llamabeast:

Good to know, my game will be for 24 players, so it helps :)

P.S. Check your PM [well, 3 from me] :)

[llamabeast]

I ingeniously increased the maximum defined number to 24 as well. :)

[Zapmeister]

It looks like something's up with the llamaserver. The last updated time precedes the current time by two and a half hours.

[Illuminated One]

It does send confirmation Emails though.

[Aethyr]

In Momentum all turns are in, but it appears it has been six hours since the last "update".

[Zapmeister]

In Momentum all turns are in, but it appears it has been six hours since the last "update".

Yeah - I'm guessing that the email server (that receives and acknowledges orders) is fine, as is the webserver. But the Dominions server itself is down and, if so, then no turns will run until llamabeast gets a chance to look at it.

[Fakeymcfake]

Has anyone tried for force host the games that have all the turns sent in?

[Tifone]

OMG the CURSE again :shock:

[fungalreason]

I believe our game probably caused it to crash http://www.llamaserver.net/gameinfo.cgi?game=Forgery. The hosting time is the same as when it stopped updating.

[Zeldor]

In Forgery I sent my turn in for last turn, I got confirmation email but my next turn says I staled.

[llamabeast]

I think then that Forgery probably crashed it. I'll take a look later today. What happened was that some instance of dom3 (let's assume it was the hosting of Forgery) took about 10 hours, before I killed it this morning. Obviously it got stuck. While there's an instance of dom3 running, no others are allowed to start, so the server jams up.

Hopefully all non-Forgery games are now back to normal.

[Zeldor]

The Curse is spreading!

[lch]

That sounds more like Horror Seed.

[vfb]

Llamaserver looks like a game crashed:

Last updated at 00:55 GMT on Sunday December 28th
Current time: 02:55 GMT

[Slobby]

New issue; I just postponed the 2 games I'm admining by 24 hours, they were both extended by ~46hours.

[DonCorazon]

http://farm1.static.flickr.com/50/138496905_acf2118171.jpg?v=0

Is the llama down?

Last updated at 13:30 GMT on Sunday February 1st
Current time: 15:55 GMT

[Redeyes]

Seems like it, it's not sending out turns either/quick host isn't working (http://www.llamaserver.net/gameinfo.cgi?game=Rothfuss)

[ano]

Also, map\mod browser doesn't work properly. Previews and descriptions are not loaded.
Perhaps, I'll have to hesitate creating a new game...:(

[llamabeast]

Great picture DC.

It wasn't happy for some reason, but I've revived it.

The map/mod browser is being intermittent, because sometimes when people upload maps/mods, I think from Windows, it kills it by inserting funny characters in its data files. For the moment I keep being lazy and simply deleting the characters rather than booting into Windows to test and fix it.

[ano]

Thanks!)
Does it mean that though it doesn't work properly, everything should be fine in game creation process?

[pyg]

Our game 'Rothfuss' just sent out turn files for turn 19. I did mine and submitted it to get back: I received an e-mail from you, but unfortunately there seems to be a problem with it.

Your 2h file seems to be for a different turn number to the one the game's currently on. The 2h file is for turn 19, while the game is now on turn 18.

Details of your e-mail:
Subject: Re: New turn file: Rothfuss, Pangaea turn 19
Sent at: Sun, 01 Feb 2009 12:12:09 -0600

Also the game status (http://www.llamaserver.net/gameinfo.cgi?game=Rothfuss) page seems to suggest the game is on turn 18 although the scores look right for turn 19.

Does this mean I get another shot at turn 18?:D

[JimMorrison]

2 games (that I am in) which hosted in the last 3 hours both had errors and gave everyone Stale Turns.

These were Chlorolera and Divine Might.

I don't know if it means anything, but it seems a bit unusual, and since the server was sad..... I dunno. :o

[vfb]

The llama web server appears to be operating in double-secret probation mode:

Forbidden

You don't have permission to access /gameinfo.cgi on this server.

Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.

[calmon]

Yep some trouble on the webserver, but the rest seems to run fine. My turn was accepted.

[rdonj]

Most curious! Upon checking my email today, I found an email about the joker2 game that goes a little like this: Hi. On trying to host Joker2 just now, Dominions crashed. This is not a LlamaServer issue, but rather Dominions itself crashed with a "Nagot gick fel" error.

Now, the reason this is odd is I'm not in joker2, nor to my recollection have I ever even subbed for the game.

And this right on the heels of that crash trying to warch a battle replay, though I'm pretty sure that's not the llamasever's fault.

[Illuminated One]

Llamaserver says there are no games running.
Also it says "Last updated at ."

[archaeolept]

may be down. has not updated in an a while now.

[JimMorrison]

I just sent a turn in a bit ago, and got a standard confirmation Email - and the game's page shows me as received. Very odd though, games do not seem to be hosting.

[llamabeast]

Yep, it got stuck waiting for something to finish hosting that never did - presumably some game crashed? Anyway, I've set it going again now.

[Zeldor]

Hmm... there is some mess with Preponderance. It was showing hosting as in the past, so I used force host. But it seems that it skipped one turn, instead of turn 56 we have 57. And it was a stale turn, but orders for turn 56 were executed. It looks like server generated turn 56, but didn't send it to anyone and didn't refresh status page. Force hosting made it host turn56 without anyone submitting it [as they were not able to do so].

[llamabeast]

Will investigate in the morning.