Tech CAguns.net Engineering

[ikeo1]

The past couple of weeks have been a whirlwind since we launched our website. Originally operating under a different domain, we made the transition to our current domain just 10 days ago. I'd like to share some insights and the motivations behind dedicating my efforts to this platform.

Like many of you, I have a deep appreciation for Calguns, despite its technological shortcomings. Recognizing the need for a fresh and improved platform, I started creating our site. The data from our brief operation suggests a strong demand for a modernized solution. Since migrating to an optimized platform a week ago, we've observed a consistent increase in traffic, indicating genuine interest.

An important aspect I want to highlight in this post is our site's efficiency, with 63% of its content being cached. This approach significantly enhances performance and user experience by reducing the direct load on our database. Our experiences with CG have informed our technical strategies, leading us to alternative forum software and architectural designs aimed at avoiding similar pitfalls.


Our infrastructure employs multiple caching layers to ensure smooth operation. We utilize Redis for session storage and Memcached for public page data, supported by CDN edge caching and extended browser cache policies for static content. Further database query caching minimizes direct SQL calls.

For content delivery, we rely on object storage and CDN rather than local storage for images, allowing for faster page loads with subsequent image loading. We've limited image sizes to 1280x960 pixels, which suits most devices, and are considering a shift to the .webp format for enhanced compression without sacrificing quality.

Our backend is designed for scalability, prepared to dynamically adjust storage, memory, CPU, and balance loads according to demand. Although it's early days, such foresight in planning speaks volumes about our commitment to growth and efficiency.

I extend my gratitude to the r/CAguns Reddit community for their invaluable feedback, encouragement, and support. The future of this platform lies in the hands of the California 2A community, and I am committed to leveraging the latest technology to serve our community without the constraints of major tech oversight.

Please let me know in the poll if this type of content is something that you would like to see and hear from. If so I'll add different aspects over time as the site develops.

 

[D00DST3R]

I love data like this. Although this isn't an area of my expertise, the fact that the information is available and shared with the community is great and helps people understand how we are doing as a whole

 

[Nyghthawk]

What about not hosting images/content outside Sell/Ad, etc. Force the use of third party, ie. Imgur, etc. This could reduce on the needs of the server as it grows. But keeping the "Sensitive" areas locally.

 

[doser]

What about not hosting images/content outside Sell/Ad, etc. Force the use of third party, ie. Imgur, etc. This could reduce on the needs of the server as it grows. But keeping the "Sensitive" areas locally.

That’s not a problem right now. We’re not serving a ton of data yet so it’s ok. Eventually what we’ll have is different layers. The free tier can store 1 cover photo and link out while contributors can upload multiple images directly. Since storage and bandwidth costs are a consideration I think this would be the most fair approach. You can still submit images but not at my expense. Then offer a filter based on that. This isn’t til later though and I will likely grandfather existing users of the platform once it happens with a limit of 3 images.

 

[D00DST3R]

That’s not a problem right now. We’re not serving a ton of data yet so it’s ok. Eventually what we’ll have is different layers. The free tier can store 1 cover photo and link out while contributors can upload multiple images directly. Since storage and bandwidth costs are a consideration I think this would be the most fair approach. You can still submit images but not at my expense. Then offer a filter based on that. This isn’t til later though and I will likely grandfather existing users of the platform once it happens with a limit of 3 images.

What about offering a membership tier system, with allotments at each tier? Helps support you and your work as well as offsets some or all the costs for the data used? Idk, just an idea thrown out there

 

[ikeo1]

What about offering a membership tier system, with allotments at each tier? Helps support you and your work as well as offsets some or all the costs for the data used? Idk, just an idea thrown out there

That's exactly what I'm talking about, but I'm on there yet. There's still more to refine and build. Charging tiers now is too early. Once we gain momentum we'll switch, but those who come in early will be grandfathered as a show of appreciation for helping to build the community. Right now the costs are low enough for me to cover, but that can change over time. It'll be defined as users up until a certain date will get that benefit. After that members will get more capabilities. We don't want to fall into the same trap as CG where it becomes unsustainable.

 

[D00DST3R]

That's exactly what I'm talking about, but I'm on there yet. There's still more to refine and build. Charging tiers now is too early. Once we gain momentum we'll switch, but those who come in early will be grandfathered as a show of appreciation for helping to build the community. Right now the costs are low enough for me to cover, but that can change over time. It'll be defined as users up until a certain date will get that benefit. After that members will get more capabilities. We don't want to fall into the same trap as CG where it becomes unsustainable.

Agreed, it's way too early for that idea but it will come to a point when the financial burden becomes difficult. If users are benefiting and enjoying the community they built and participate in, they should at least be open to the idea of contribution

But that bridge will be crossed at that time

 

[RustyIron]

Thanks for building the new site. I've been a member for only a couple days, and I think this is my first post. You guys probably have your ideas on how you'll set things up, but if you're seeking input, I have a couple observations/thoughts.

1. Your new classifieds format looks good, despite my not having used it yet. My big gripe with the other site is that sellers could "bump" their item daily. That resulted in hundreds of redundant listings, making navigation cumbersome.

2. I don't think remote image hosting is a good idea. Hosting sites come and go, url structure changes, and users accounts fade away. One need only look to other forums that require remote images. Old threads on those forums are filled with gray "Image not found" boxes. Maybe the cost of keeping images on this site is prohibitive, but the long term relevancy of threads is something to consider.

 

[ikeo1]

1. Your new classifieds format looks good, despite my not having used it yet. My big gripe with the other site is that sellers could "bump" their item daily. That resulted in hundreds of redundant listings, making navigation cumbersome.

For now this isn't a problem due to how few there are. I know what you mean. It's pretty annoying. I've seen other sites do a tiered system where supporters who contribute financially are able to bump more often than free tier. Free tier, once a week. tier 1 once every 3 days tier 2 once a day.

2. I don't think remote image hosting is a good idea. Hosting sites come and go, url structure changes, and users accounts fade away. One need only look to other forums that require remote images. Old threads on those forums are filled with gray "Image not found" boxes. Maybe the cost of keeping images on this site is prohibitive, but the long term relevancy of threads is something to consider.

The way I envision, free tier will get 1 image so there will always be something to show. The listings also expire after 90 days, so there may be some room to manage that too. I definitely want to clean up the clutter and offer a better experience for people on both sides. Free sounds good, but it comes with a different kind of cost. I'll be putting more though into it as I build out different tiers of membership. We'll also have vendors/verified sellers etc.

Thank you for the suggestions. It leads me to think we're heading in the right direction with the right priorities. None of this is going to be put in place at the moment but will gradually be implemented for free until we start accelerating in growth and we have to make some adjustments.

All of this will be communicated in the forum first for feedback and comments.

 

[yatzooa]

Thanks for sharing these insights, love to see the data behind magic! Any chance we could also get metrics on total threads and/or posts created per day? That'd be a fun metric to observe as the website grows, since that would be a good indicator of forum activity

 

[ikeo1]

Thanks for sharing these insights, love to see the data behind magic! Any chance we could also get metrics on total threads and/or posts created per day? That'd be a fun metric to observe as the website grows, since that would be a good indicator of forum activity

When they start increasing I’ll post more metrics. This isn’t going to be a frequent thing but I will provide regular updates on different things.

The main thing is that it’s too low right now to be interesting, I may be wrong, but once we get some traction I’ll be glad to do a series on it.

 

[Error-404]

The transparency about the technical performance says a lot about how much you care for this community, and we appreciate it as well.

 

[Easye240]

love the site so far. How are you spreading awareness? This is a LOT better than Calguns in its current state

 

[ikeo1]

love the site so far. How are you spreading awareness? This is a LOT better than Calguns in its current state

Every now and then I post on r/CAguns to get feedback on the next set of features. I get a wave of new users and rush to adjust the welcome experience. I'll probably post there less while I improve the onboarding experience. The big updates are mostly done where most of the input is not about what's missing anymore. I think I got most of the functionality covered.

Other than that, I'm not really spreading it other than one post in a FB group I'm in. I'm hoping word will spread organically, through current users, while I focus on cleaning up various parts of the system while also setting up other parts of the site like Reviews and Resources.

I'm still actively working on it, and we get new people every day. If some bigger platform mentions us and we get more engagement, I think that's when things will really grow. For now, I'm trying to prepare for growth and make caguns.net the way people think a website like this should be.

Thanks for commenting and welcome aboard! Keep an eye on this forum cause this is usually where I ask for feedback on how we should set things up. The more people who contribute, the better I can prioritize. The reddit sub was really good at what they decided was important. I want to keep that going.

 

[couragewolf]

The past couple of weeks have been a whirlwind since we launched our website. Originally operating under a different domain, we made the transition to our current domain just 10 days ago. I'd like to share some insights and the motivations behind dedicating my efforts to this platform.

Like many of you, I have a deep appreciation for Calguns, despite its technological shortcomings. Recognizing the need for a fresh and improved platform, I started creating our site. The data from our brief operation suggests a strong demand for a modernized solution. Since migrating to an optimized platform a week ago, we've observed a consistent increase in traffic, indicating genuine interest.

An important aspect I want to highlight in this post is our site's efficiency, with 63% of its content being cached. This approach significantly enhances performance and user experience by reducing the direct load on our database. Our experiences with CG have informed our technical strategies, leading us to alternative forum software and architectural designs aimed at avoiding similar pitfalls.

View attachment 297
Our infrastructure employs multiple caching layers to ensure smooth operation. We utilize Redis for session storage and Memcached for public page data, supported by CDN edge caching and extended browser cache policies for static content. Further database query caching minimizes direct SQL calls.

For content delivery, we rely on object storage and CDN rather than local storage for images, allowing for faster page loads with subsequent image loading. We've limited image sizes to 1280x960 pixels, which suits most devices, and are considering a shift to the .webp format for enhanced compression without sacrificing quality.

Our backend is designed for scalability, prepared to dynamically adjust storage, memory, CPU, and balance loads according to demand. Although it's early days, such foresight in planning speaks volumes about our commitment to growth and efficiency.

I extend my gratitude to the r/CAguns Reddit community for their invaluable feedback, encouragement, and support. The future of this platform lies in the hands of the California 2A community, and I am committed to leveraging the latest technology to serve our community without the constraints of major tech oversight.

Please let me know in the poll if this type of content is something that you would like to see and hear from. If so I'll add different aspects over time as the site develops.

I'm a TPM that's worked at a number of FAANGs remotely in LA and I just want to give you a solid round of applause for all of this

If you ever want to talk shop I'm here for it.

 

[ikeo1]

Just a follow up on the cache savings. After finding a bug in the caching algorithm and fixing it a few days ago. There's been a noticeable improvement in terms of our content serving. We've gone from averaging ~63% to 85% from cache in our best case scenarios. This means that the database only see's 15% of the actual traffic that visits the site for initial loads and changes.


This results in a huge savings for us and also a performance boost for all users. We were able to improve end user performance while reducing workload on the backend. I've also reduced the maximum size of images to 1024 which should be sufficient vs 1280 to reduce average file sizes to < 100kb per image resulting in smaller payloads. Once we get .webp, we may be able to increase back to 1280 while keeping the smaller file sizes.

We haven't had the need to really do database optimization yet, but once we get more traffic, that's where we'll be exploring next. As of now, there's not a lot of load on the database and our search engine is another place we can do additional tuning. Right now, there's little need to tune. Currently we're working on additional documentation to help onboard new members. And cleaning up different pages for easier use.



At any given time we serve thousands of requests per hour, eventually it'll matter more when we get to hundreds of requests per second. There's basically no load on our servers. The database and search engine are even lower utilization. Granted, the amount of traffic is still relatively small. You can see that our architecture is ready for surges in traffic. We're as performant as we need to be for now.

 

[Chupacabra]

Does this new site have or need a backup site? I ask because the other .net site has a .org backup site as it undergoes a DB backup. I know vBulletin is ancient compared to the modern features of XenForo.

 

[ikeo1]

Does this new site have or need a backup site? I ask because the other .net site has a .org backup site as it undergoes a DB backup. I know vBulletin is ancient compared to the modern features of XenForo.

I have daily offsite backups and we have a discord, but not opening it up yet until things grow a bit more. Also I would do an update on YouTube. I’m planning to do a series on the rise of caguns and a post mortem of the calguns migration. I’ve experienced what’s happening to them early in my career.

TBH they should’ve rolled back and let the dev team try to restore the site with a backup copy vs trying to do it realtime.

 

[ikeo1]

Another interesting check in. The growth rate is accelerating. The first post took metrics across 7 days. Due to the recent issue at Calguns, we're seeing a really big growth of users.


If we take a look at today's metrics we see that what took 7 days is now almost equal to a 24 hour cycle. Our cache rate improved to the point where the majority of data is cached.



Fortunately, we're serving less data for the same traffic. Optimization is working. The number of unique's that reach the site is growing. On March 26/27, we migrated to our current provider. So we can see how data is trending. We've had 27k unique visitors in 3 weeks on the current platform and nearly 2 million requests. It was only a few days ago where we just reached our first million requests. What this says is that as the content keeps building up, people are staying longer and longer. This is backed up by our other analytics data which shows that an average user stays for about 6 minutes going through the site. Functionality will be less and less of a priority and building up the content will be the next priority. Populating the different areas of the site to continue to build up on that number.



Stay tuned for the next installment of seeing how the site continues to grow. It's fun to keep tabs on this data and see how things we did early on help us as the site continues to accelerate. CG will come back online eventually, but hopefully word is getting out fast enough for people to come check out this site. Once they visit us and then end up going back to CG, it'll be somewhat of a shock, which is what we're going for. By then, hopefully momentum picks up steam and our growth won't slow down too much once they come back online. We're ready to go and will continue to do patching as needed. We'll have scheduled maintenance windows to make sure our systems function the way we expect during an upgrade.

 

[ikeo1]

4/29/24 Progress Report​

We're back just a week later, and what a difference a week can make. I like to keep these updates rolling to track the progress of our site—it’s been quite the adventure with significant activity from week to week. Fortunately, thanks to our earlier efforts, we’ve managed to onboard all the new users smoothly. Our documentation held up well, making the process relatively consistent and less chaotic despite the surge in new listings. Users are becoming more familiar with the site, bringing more consistency to their interactions as we continue refining our guides.

The chart below illustrates that in just 7 days, we've surpassed what was achieved in the previous 30 days. Now that the caching rate has had a chance to stabilize over a week, it’s hovering just shy of 87%. This means we only need to hit the database or origin servers about 13% of the time, primarily when users upload pictures or create new posts and catch up on threads.



With the uptick in activity, bandwidth usage has increased by ~30% week to week and ~100% compared the previous 4 weeks. As we welcome new users, each one tends to add multiple images to their account, including avatars, posts, and classifieds. This compound effect—as people add images and review the classifieds section, accounts for the projected increase in bandwidth usage. It’s important to set expectations for limits early on, as reflected in our analytics, which show a significant increase in engagement time. As more people sign up and contribute, everyone tends to stay on the site longer. We're committed to nurturing this behavior and plan to introduce features to sustain certain activity levels. Managing the user experience remains a top priority, and while we can't perfect everything with a small team and a demanding niche, we strive to deliver the best possible service for the majority.



Most of our community now understands that we may throttle bandwidth as necessary to support continued growth. With the recent influx of users, I've identified several areas for improvement, and we'll be conducting some maintenance later this week anticipating future waves of users. We're also planning to restrict more resource-intensive services to registered users only. Currently, the ratio of guests to registered users is 2:1, and we aim to at least balance this to 1:1 as we prepare to welcome more members.

Our growth over the last 30 days has been impressive and underscores our potential. We're momentarily pausing to ensure stability, but I anticipate continued growth as more people learn about the user experience we offer. Over time, we will likely gate more resources to ensure the sustainability of our services. Adding features requires significant computing power and bandwidth. Our restrictions aim to maintain a consistent and high-quality user experience. So far, the community has responded positively to these measures, especially since we’ve set these expectations from the start. We're currently looking for sponsors and advertisers, which takes time to build and set up because of the nature of the 2A space and limited advertisers. It's been surprisingly difficult in this industry compared to other industries I've worked in. No worries, we'll find good partners who want to build this growing 2A community with us.

With that, I'd like to thank the CAguns Supporters because every donation or membership helps offset the personal cost for me to provide this service to the 2A community. I feel like it is worth the investment and that things will work out in time as the community grows. We have a good runway, but it depends if things run-a-way.



This journey is just a fraction of what’s possible compared to the broader community. A 10x growth, while ambitious, is entirely within reach, and we may need to make further adjustments as more users join. In fact, we've already grown 10x in the last couple of weeks. I'm excited to see the system being tested so thoroughly early on—it really validates that we have the right product at the right time.

I hope the site works well for you and Thank you for helping to build the next 2A space for the California community.