Poll Calguns Data Migration Round 2 - Aug 3

[EBR Works]

Thanks so much to the owner/admins of this site! Having been a vendor on CalGuns for about 15 years now and member since 2008ish, it's really frustrating to pay Kes around $1200 per year for vendor status and have the site essentially never work, for months and months. I fear that it's never coming back..

Anyway, great to be here!

 

[Malibu]

Thanks so much to the owner/admins of this site! Having been a vendor on CalGuns for about 15 years now and member since 2008ish, it's really frustrating to pay Kes around $1200 per year for vendor status and have the site essentially never work, for months and months. I fear that it's never coming back..

Anyway, great to be here!

I would guess that there is no further recourse, and as a result from this point forward no longer paying for non-performance. Can that “vendor status” expense be somehow used for your tax purposes?…If so, call it a lesson in advertising.

 

[ikeo1]

Thanks so much to the owner/admins of this site! Having been a vendor on CalGuns for about 15 years now and member since 2008ish, it's really frustrating to pay Kes around $1200 per year for vendor status and have the site essentially never work, for months and months. I fear that it's never coming back..

Anyway, great to be here!

You're welcome and Glad to have you here! Thank you for helping to support the site too.

I'd be frustrated too if i were paying that much with the site in its current condition. Ironically, people are willing to throw more money into it while trying to get work done for free. I'm not sure what the future for that site holds, but I think for us we're on a good path.

I hope we can support more businesses and the community to get engaged in the sport again.

 

[Radman64]

Whatever Kes tried to do he totally botched it, trying to go there for me it is nothing but delays for anything to load and always at some point it won't load.
Still makes me upset it shows me as joining in 22 when I joined in 08/09, was a number of years I never logged in but when I tried it did not recognize my password yet my email addy showed I had an account, used the same password I always have for sites like this and I was good to go again.
My history was wiped out and I was listed as a new member, when I reached out about it asking why nothing happened doing what I was told to do lol.
The guy ruined the site imo.

 

[ugimports]

Whatever Kes tried to do he totally botched it, trying to go there for me it is nothing but delays for anything to load and always at some point it won't load.
Still makes me upset it shows me as joining in 22 when I joined in 08/09, was a number of years I never logged in but when I tried it did not recognize my password yet my email addy showed I had an account, used the same password I always have for sites like this and I was good to go again.
My history was wiped out and I was listed as a new member, when I reached out about it asking why nothing happened doing what I was told to do lol.
The guy ruined the site imo.

It's strange..that happened to a handful of users as I've read..I wonder what happened...I would have expected something like that to be an all or nothing sort of issue.., I joined around the same time you did so it wasn't as if it was everyone from 2008-2009...

 

[Malibu]

I think I’ll try to register over there as a new user with the name of “504 Gateway Timeout”

 

[ikeo1]

I feel bad for the folks thinking the site is getting faster when its really just not as many people visiting and slows down during traffic hours. Hopefully we get updates going but it doesn't seem like transparency is a thing.

What a lot of the folks there don't realize is, as folks use this site more, CGN speeds up. So if this site did not exist, it would be a ton worse. We're offloading traffic and that helps the die hards there.

 

[Kyle1886]

I feel bad for the folks thinking the site is getting faster when its really just not as many people visiting and slows down during traffic hours. Hopefully we get updates going but it doesn't seem like transparency is a thing.

I don't think it so much about traffic of members. This AM and like previous AM's I try to get on site around 0245 to 0300 hrs daily. There is very little traffic at that time. The last few mornings nothing but 504s on any attempts to get on. This AM 0258 hrs to 0714 hrs nothing but 504s or an occasional 502.

Up until a couple of weeks ago, I could get on at those times, but get 504ed between 0315 and 0320 hrs as if something is timed to run at that time, thusly bogging down to our beloved 504.

Just an observation, nothing more.

Respectfully
Kyle

 

[ikeo1]

I don't think it so much about traffic of members. This AM and like previous AM's I try to get on site around 0245 to 0300 hrs daily. There is very little traffic at that time. The last few mornings nothing but 504s on any attempts to get on. This AM 0258 hrs to 0714 hrs nothing but 504s or an occasional 502.

Up until a couple of weeks ago, I could get on at those times, but get 504ed between 0315 and 0320 hrs as if something is timed to run at that time, thusly bogging down to our beloved 504.

Just an observation, nothing more.

Respectfully
Kyle

It's a bit of both. I've seen your posts and wanted to mention it but knew the folks there would give me a hard time.

vbulletin runs periodic jobs, so its all database contention. The reason its random is because there's bad queries and likely bad data the system is chugging through. The AM is likely backup times as I have a similar backup window and is pretty standard. At other times, its likely there's a combination of users using different resources that causes the contention.

There's a few different scenarios that cause the issue.

• Forum is under heavy load with readers with people refreshing the site
  • concurrently people are posting in the marketplace
  • file sizes also too large with attachments
    • Some folks may notice here, sometimes we remove the full size image in place of the thumbnail. We're testing to see the impact but prefer folks to use thumbnails. This makes the request onClick vs onLoad saving you and us bandwidth unless you actually want to see the image.
    • It also makes it a gallery experience vs a scrolling experience, which we think is cleaner.
• Marketplace views overloading connections
  • file sizes are too large when a large number of users view/posting this drains connections
• Search
  • If any of the combination above while someone is searching the system and does a table scan because its vbulletin

If you've read the migration plan here, then you know this is what we plan for and how to mitigate it. I look at the problems on the site and engineer around them. We've learned all the lessons we need to when it comes to issues with scaling and data management. A lot of it has to do with the frequency of posting too. Which is why the more people come here the faster CGN gets because there's less contention in the 3 areas.

 

[-D.M.-]

I don't think it so much about traffic of members. This AM and like previous AM's I try to get on site around 0245 to 0300 hrs daily. There is very little traffic at that time. The last few mornings nothing but 504s on any attempts to get on. This AM 0258 hrs to 0714 hrs nothing but 504s or an occasional 502.

Up until a couple of weeks ago, I could get on at those times, but get 504ed between 0315 and 0320 hrs as if something is timed to run at that time, thusly bogging down to our beloved 504.

Just an observation, nothing more.

Respectfully
Kyle

There's a guy whose YouTube I subscribe to named Ryan Macbeth, he's pretty sharp. One of the ways he tracks russian disinfo/misinfo is by the common "work" times in russia. It seems if this site is any indicator, the vast bulk of bad actors clogging up the traffic come from russia. So if you were to say, look at what times the site is the fastest versus when it's slowest, I wonder if you'd see it corresponding to average waking hours in russia or china.

 

[H. Bowman]

There's a guy whose YouTube I subscribe to named Ryan Macbeth, he's pretty sharp. One of the ways he tracks russian disinfo/misinfo is by the common "work" times in russia. It seems if this site is any indicator, the vast bulk of bad actors clogging up the traffic come from russia. So if you were to say, look at what times the site is the fastest versus when it's slowest, I wonder if you'd see it corresponding to average waking hours in russia or china.

I very much doubt that Russia cares about calguns.

 

[ikeo1]

I very much doubt that Russia cares about calguns.

The reason he says this is due to what was mentioned in the private subscriber channel. Russia is by far the biggest aggressor to our site. I've had several sign up attempts from there in the past and one of the main reason why we have some of the policies we have. This sample is within the last 24 hours. I'm not sure what they would want on here. I can only assume they tried to sign up and continue to try to access the site via bots and crawlers. Once you put it in blocks, its interesting to see how many members start to contact you saying they're blocked. I have added several layers of security with the recent changes, which is why you see our guest count is very low.

I've also limited what guests can see and focusing more heavily less visibility to the public and more so to members only over the last couple of weeks. If specific members are crawling the system, I'll see that activity.

 

[wtf_calguns_migration]

4am: DNS changes are happening.
9pm: DNS changes are still happening. Or DNS is stable, but the server isn't up.

I don't know about you, but when I have to change DNS, it typically takes me less than a minute. And propagation is pretty fast these days. Something is not right here.

Agreed, there has been a stream of statements about their troubles over the years that have not sounded right at all.

This is compounded by the fact I have met some of those people at ranges with network admins, data engineers, server jockeys. Let's just say that there are some immediate and simple things that continually confound them. They don't speak to anyone that work at ISPs, platform providers, application publishers, etc.

 

[wtf_calguns_migration]

I very much doubt that Russia cares about calguns.

CG migration and down statements sound like they're ddosing themselves more than not.

 

[wtf_calguns_migration]

Yeah server isn’t up. Not sure why they would change dns without it up. Its just like before bit

If you want your data secure use caguns.net The security protocols between the two sites couldn’t be more different.

View attachment 5454

I know some folks want to be nostalgic but man, I don’t want to be that guy. This isn’t the way to do these kinda things.

We’ll have a migration coming up testing out our backup and upgrade process. Hopefully offering some new features in the future.

Alias pointing to compute that isn't up is messy. Claiming they cannot extract data, push to holding tables, new data tier ingests old data is laughable. That's literally every data product company or data service reseller ever. Replatforming is so common bc datacenter drive manufacturers increase iops and capacity every 3-5yr refresh window that it's jr level junk work in migrations.

I guess if they wanted to go back to bare metal, fine. They aren't talking about anything more complex than any company doing a common gear refresh. You'd normally get a massive jump in cores and ram density next but. A single host in the new 3pack should reflect as much of not more capacity than the old cluster.

There lies the rub, bc the statements made by them over the years reflects a very simple mindset. It's a box hosted somewhere. It's not rows in 3+ locations that serve users in the densest user pools by geo. There's a distinct lack of basic WAN network maintainance, much less HA/FT/DR thoughts.

They've had time to learn, and it smells a lot like they paid junior college level desktop interns the whole time.

 

[Malibu]

What is meant by “crawling the system”?

 

[ikeo1]

What is meant by “crawling the system”?

There are many different kinds of bots doing different actions. Some better than others. I review all the bots that crawl the system and have gated several parts of it to prevent user information from getting out.

One change coming is not to list phone numbers in listings. We want people to maintain a level privacy and move communications directly. If your phone number & email is listed its crawled by several bots and engines. There's never a good reason for a bot to have your email and phone number.

https://www.cloudflare.com/learning/bots/what-is-a-web-crawler/

 

[wtf_calguns_migration]

What is meant by “crawling the system”?

You can traverse thru a system, you can collect data commonly called "scraping". People tend to use terms imprecisely.

I do not want unauthorized traversals in my system. I could see benefits to allow limited public scraping and/or indexing.

There is a use case for most things.

 

[wtf_calguns_migration]

I feel bad for the folks thinking the site is getting faster when its really just not as many people visiting and slows down during traffic hours. Hopefully we get updates going but it doesn't seem like transparency is a thing.

What a lot of the folks there don't realize is, as folks use this site more, CGN speeds up. So if this site did not exist, it would be a ton worse. We're offloading traffic and that helps the die hards there.

Depends on your caching and CDN tuning. If a half petabyte of data can be ingested and propagated to edge targets in about 30secs within a Region in a CA footprint B2B provider....then it ain't rocket surgery.

Also when all the other engineering bodies pick up enough network admin to cover CCIE concepts as a daily within 2 years then there is no excuse for CGs problems.

Then again I've actually scripted HA/FT/DR tests and have them signed off successfully after completion. Supposed to have done that with environment setup as an automated process if your employer has any decent customer velocity.

 

[-D.M.-]

I very much doubt that Russia cares about calguns.

Shortsighted thinking like "what would russia want with a small local gun forum" is probably why calguns is having so many damn problems. Because clearly, there's interest. Or maybe the simple inability to consider that while there's information to scrape from calguns, sometimes it's as simple as establishing a "backstory" or seemingly innocuous history for when they go for their real targets.