Posts: 1,749
Threads: 7
Joined: Oct 2013
Reputation:
3
Bob Schroeck Wrote:Once things shake out here financially -- see other thread in this forum for one of several reasons why -- I do intend to contribute as well.
One question, though... why are we getting 403 Forbidden errors now?
That's related to the 504's. When the server starts to gag, we get 504's. When it reaches TILT! levels (it can't even do anything because it's out of juice), the 403's happen.
Unfortunately, this will be an ongoing occurence until we can get this resource issue under control.
Posts: 25,622
Threads: 2,062
Joined: Feb 2005
Reputation:
12
What kind of hardware are we looking at? (And what OS - Win, *nix, AIX, other?) While we're at it, does the server farm need a NAS/SAN upgrade at the same time?
May as well nail down a ballpark figure before we start the fundraising drive - give ourselves a goal to aim at...
--
Rob Kelk
"Governments have no right to question the loyalty of those who oppose
them. Adversaries remain citizens of the same state, common subjects of
the same sovereign, servants of the same law."
- Michael Ignatieff, addressing Stanford University in 2012
Posts: 1,749
Threads: 7
Joined: Oct 2013
Reputation:
3
robkelk Wrote:What kind of hardware are we looking at? (And what OS - Win, *nix, AIX, other?) While we're at it, does the server farm need a NAS/SAN upgrade at the same time?
May as well nail down a ballpark figure before we start the fundraising drive - give ourselves a goal to aim at...
The Orain server cluster currently runs on Ubuntu 14.01 LTS x86-64. Our current issue is that we have too few servers to handle all the tasks in question (we have 6-8 at present, and some servers are currently handle a disporportionate amount of processes), so we are looking into getting two more, which wiill cost $5 a month per server for upkeep/maintenance costs.
Ther new servers will be used to spread around the various processes used by Orain to keep their server setup going and redistribute the load balance so no one server is handling most of the work, wihich is the source of the issue.
To my knowledge, no additional hardware is also required at this time.
Posts: 25,622
Threads: 2,062
Joined: Feb 2005
Reputation:
12
Just $5/month/server? And getting some will speed up my wiki on Orain as well, as a knock-off effect? I think I can fund one server on a regular basis. (Have to step away from the computer for a while. If you email me with the details on how to contribute - my board handle at gmail - I can get something set up once I'm back.)
That leaves one server (or more) for everyone else to fund...
--
Rob Kelk
"Governments have no right to question the loyalty of those who oppose
them. Adversaries remain citizens of the same state, common subjects of
the same sovereign, servants of the same law."
- Michael Ignatieff, addressing Stanford University in 2012
Posts: 1,749
Threads: 7
Joined: Oct 2013
Reputation:
3
robkelk Wrote:Just $5/month/server? And getting some will speed up my wiki on Orain as well, as a knock-off effect? I think I can fund one server on a regular basis. (Have to step away from the computer for a while. If you email me with the details on how to contribute - my board handle at gmail - I can get something set up once I'm back.)
That leaves one server (or more) for everyone else to fund...
Check the Orain Meta front page, click the Donate button, will tell you all you need to know.
Posts: 3,710
Threads: 95
Joined: May 2012
Reputation:
9
It depends on the server, more or less. But see pricing here: http://www.digitalocean.com/pricing/
Naturally, some boxes want more space; some want more memory but they don't really have configurations like that so I believe 2 of them are on the $10/month plan. VPS are so cheap these days, one of these days I might want to spin up an instance for myself.
-- ∇×V
Posts: 27,639
Threads: 2,274
Joined: Sep 2002
Reputation:
21
Today's random annoyance: disappearing images.
I feel like the guy who animated the "Today's mission: FAILED" at the end of every episode of Excel Saga...
-- Bob
---------
Then the horns kicked in...
...and my shoes began to squeak.
Posts: 1,749
Threads: 7
Joined: Oct 2013
Reputation:
3
Bob Schroeck Wrote:Today's random annoyance: disappearing images.
I feel like the guy who animated the "Today's mission: FAILED" at the end of every episode of Excel Saga...
This is explainable.
Basically, we're shuffling around the responsibilities of each server so no one server will be handling too much of a process load, and we're having to experiment some to find a happy medium, as on top of getting new servers, we want to optimize the load balance of the existing servers.
The end result should mean greater uptime and performance for every aspect of the wiki farm.
Posts: 27,639
Threads: 2,274
Joined: Sep 2002
Reputation:
21
Ah, okay, that's very reassuring! Thanks!
-- Bob
---------
Then the horns kicked in...
...and my shoes began to squeak.
Posts: 4,891
Threads: 302
Joined: Jul 2010
Reputation:
8
Sounds like five people juggling ten footballs between them.....
________________________________
--m(^0^)m-- Wot, no sig?
Posts: 25,622
Threads: 2,062
Joined: Feb 2005
Reputation:
12
So much for doing a quick bit of editing before work... I can't log in.
--
Rob Kelk
"Governments have no right to question the loyalty of those who oppose
them. Adversaries remain citizens of the same state, common subjects of
the same sovereign, servants of the same law."
- Michael Ignatieff, addressing Stanford University in 2012
Posts: 27,639
Threads: 2,274
Joined: Sep 2002
Reputation:
21
I never log out if I can help it, so I can't confirm the problem, but I will note that I got to the ATT front page (still properly logged in) with no problems less than a half-hour after you posted that.
EDIT: I just noticed -- the main menu along the left edge is severely reduced in size and options. Something is (as usual) going on again.
EDIT 2: Confirmed. I could not save at all because of "session information lost", and error message suggested logging out and logging back in again. I usually ignore and retry and that's enough, but after a half-dozen failed retries I went for the logout -- and ended up with the same cookies message. So... if the save failure is universal, and the recovery advice results in an inability to log in and be able to edit, then the entire wiki is read-only right now. Joy.
-- Bob
---------
Then the horns kicked in...
...and my shoes began to squeak.
Posts: 27,639
Threads: 2,274
Joined: Sep 2002
Reputation:
21
Oh, and I'm assuming the current crop of spammers are coming from a wide variety of IPs, so that's why we're not just locking that address out.
And while I'm at it, Geth, did you mean to block the New User Message bot/script?
-- Bob
---------
Then the horns kicked in...
...and my shoes began to squeak.
Posts: 1,749
Threads: 7
Joined: Oct 2013
Reputation:
3
Bob Schroeck Wrote:Oh, and I'm assuming the current crop of spammers are coming from a wide variety of IPs, so that's why we're not just locking that address out.
And while I'm at it, Geth, did you mean to block the New User Message bot/script? As for blocking New User Message, oops.
In my defense, I was half asleep and saw spam and that's like waving red in front of a bull for me (I H8 spam with the fury of a million burning suns), so once we can edit again (I checked, database is read only even for me, so John must be doing maintenance), so I'll unblock it once that's over.
As for the database being read only, good idea why that's happening, and it's for our benefit: One, John is currently CU'ing all the spammers IP so he can range block them (not just ATT has been blitzed by these idiots, as they seem to have made it past our current abuse filters), and the read only database bit will prevent more spam from being created until he deals with these idiots, and he's also been working on getting HHVM to start on it's own server (to prevent crapouts when run on servers with other processes), and I assume he's testing that with the database in read only mode for bugtesting reasons.
Anyway, I'll see if I can't get some more info on this, should be over soon hopefully.
Edit: Our log in server is screwed up, will update when that is fixed.
Posts: 1,749
Threads: 7
Joined: Oct 2013
Reputation:
3
Fire's out now, we're good again.
Posts: 27,639
Threads: 2,274
Joined: Sep 2002
Reputation:
21
Cool.
Now if I could only stop feeling like an incessant whiner...
-- Bob
---------
Then the horns kicked in...
...and my shoes began to squeak.
Posts: 1,749
Threads: 7
Joined: Oct 2013
Reputation:
3
Bob Schroeck Wrote:Cool.
Now if I could only stop feeling like an incessant whiner...
Bob, even though I'm not an official staff member of Orain in name (though I have most of rights and duties of one), I utterly understand how you feel, and I'm glad you are so prompt in bring this sort of thing to my (and Orain's) attention, as that means we can address the problem sooner.
In this case, turns out John fell asleep at the wheel messing with settings when shuffling stuff around and that caused the log in issues, and as soon as we got the word, he was in IRC real quick to report he was on the issue, so you did the right thing.
Also, I bring good news: John just range blocked a ton of spammers, so the spam hordes should now be properly suppressed.
BONUS: We also have begun doing some major optimization, removing some processes that were uneeded but were sucking resources anyway, so we should be seeing wiki farm wide site improvement very shortly
Posts: 27,639
Threads: 2,274
Joined: Sep 2002
Reputation:
21
Okay, I feel a little better now. My thanks to you and to John...
-- Bob
---------
Then the horns kicked in...
...and my shoes began to squeak.
Posts: 27,639
Threads: 2,274
Joined: Sep 2002
Reputation:
21
Hm. Eight new users created between 1:30 and 5:30 AM this morning, all of them with names reminisicent of the spam accounts that we've had to deal with recently. Seven haven't done anything yet, but one has, and I've deleted and banned accordingly. We should keep a close eye on the rest.
-- Bob
---------
Then the horns kicked in...
...and my shoes began to squeak.
Posts: 1,749
Threads: 7
Joined: Oct 2013
Reputation:
3
We are FINALLY getting some good audio/video embedding!
10-24-2014, 07:31 PM
http://www.mediawiki.org/wiki/Extension:Widgets
Short version, it's a much simpler way to embed video/audio that doesn't require server backend nonsense, external files, or complicated setup.
Southparkfan is even going to donate a copy of his Youtube widget as a nice bonus.
Posts: 27,639
Threads: 2,274
Joined: Sep 2002
Reputation:
21
Oh, excellent! And the fact that I've been too busy (when the wiki's been functional at least) to do anything means I won't have to undo anything when this is live.
On the topic of functionality, what's up with the wiki today? I'm seeing 504s more than I'm seeing readable or editable pages, and it's been a day or so since I saw a dependably-displaying work template.
-- Bob
---------
Then the horns kicked in...
...and my shoes began to squeak.
Posts: 1,749
Threads: 7
Joined: Oct 2013
Reputation:
3
John's still working on that. Also, the search backend needs fixed on the database end of things, but that's on the list of definite to dos.
Posts: 1,749
Threads: 7
Joined: Oct 2013
Reputation:
3
WE CAN EMBED VIDEO AND AUDIO NOW!
10-24-2014, 09:39 PM
http://allthetropes.orain.org/wiki/For ... eo.21_1832
So far, we just have a Youtube widget, but it works fine.
The post linked above has instructions how to add embedded Youtube links and how to request more widgets.
Posts: 1,749
Threads: 7
Joined: Oct 2013
Reputation:
3
More good news, we're getting another server to run ATT's search engine with ElasticSearch again (now security hole free)
Posts: 3,710
Threads: 95
Joined: May 2012
Reputation:
9
"GethN7 Wrote:(now security hole free*)*until they discover the next bug FTFY
-- ∇×V
|