14 comments

  • arjie 33 minutes ago
    The biggest mistake I made was high uptime. arjie.com was up for 10 years plus on a Hetzner VPS so that by the time they wanted to sunset the machine underlying I had no idea what my teenage self had set up. I have the backups but the site hasn’t been up in a decade…

    Nowadays I build things so that they move and I have moved things about a bit so I know they work.

    • bfivyvysj 6 minutes ago
      I started putting things in a big ansible playbook repo. Don't need to have it fully managed by ansible either I mostly just have setup configured there I still do lots of by hand management.
    • nine_k 19 minutes ago
      Indeed, for a VM, high uptime makes little sense, because a reboot takes a few seconds, and an upgrade requires no downtime, just switching the DNS to a new instance.

      For a physical machine which you can't easily copy, it's a different story.

  • coreyp_1 2 hours ago
    I'm in the same boat. I have 2 old servers that I let get "too" old, and now I'm afraid to touch them to update them. However, with some of the shenanigans that the Linux distributions are pulling around age verification/attestation, I'm considering bailing on them entirely.

    Note, I did try Artix, but when it broke last week after a restart (in which evidently something had gone wrong with an earlier kernel update), and I had to pull out a rescue ISO, I decided I didn't want to mess with that. I switched that machine to Devuan, but the jury is still out for me. I don't have any major complaints, but I'm still in the burn-in phase. :) I'm running Arch on a laptop, but they have been a bit hostile in the community with censorship, so I'm just waiting for a free weekend to blast it and put something else on. I don't want political drama in my software.

    This all comes at an interesting time, though. This is the first time that I purchased a new laptop and didn't even let it boot into Windows, but instantly installed Linux. And everything "just worked". And now that I'm excited to try Linux, so many of the big players are embracing the steps to erode privacy (AI everywhere... age attestation/verification... telemetry on by default...). It's sad, and I'm just going to "nope" out of any interactions with them.

    • sgc 1 hour ago
      FWIW, I once abandoned an Ubuntu server for a decade, and managed to update it painlessly in 20 minutes. That same server is still running today, now with the latest LTS. I think I might have even started with Ubuntu 4, or perhaps 6, and it has been painless all the way through. Perhaps my slow upgrades saved me from early adopter woes :).

      I use Debian now much more. With all the supply chain attacks, Debian Stable feels like an absolute jewel, even if there are always a few packages I need to handle separately because I want or need a more updated version. But I love the old school no-nonsense engineering ethos.

    • rlpb 1 hour ago
      > However, with some of the shenanigans that the Linux distributions are pulling around age verification/attestation...

      You've been misled.

    • stock_toaster 1 hour ago
      My servers/VMs typically run either FreeBSD or Alpine. A Debian here or there where needed (proxmox, VPS that doesn't support Alpine, corp stuff, etc).

      I've also got a couple of test systems running Chimera - going to wait until it hits stable before relying on it too much though. Experimenting a bit with AerynOS too.

  • tracker1 1 hour ago
    Personally, I've been running with Caddy in front of Docker (compose) for most of my personal/hobby usage. If it's a straight website, I'll let Caddy serve the contents directly... for "web apps" I'll pretty much containerize all the things and use caddy for TLS termination and reverse-proxy duties to the app running under Docker...

    Mostly ~/apps/appname, where each appname has a docker compose file, and the data directories mounted under appname... I can compose down and (s)ftp the data out for hard archives or to move a site/service. I had been running a few VMs under a dedicated server, but switched to separate VPSes on OVH. Only gotcha with OVH is if you want to run mail, you want to avoid the local zone VMs that don't allow mail hosting.

    YMMV

  • adamddev1 1 hour ago
    I enjoyed my foray into trying FreeBSD for my personal server. There's something cool, clean, simple and "punk rock" about it. But I gave up as my main pain points were:

    - PM2 was buggy on FreeBSD, which I used to manage my processes

    - An alternative, using `rc.d` to run daemons was just so hard to get logs working.

    - The firewall required too much self configuration to get it right with all the best security practices (ie. What does one do with ICMP.) I was missing something like a template with the defaults that come with UFW, for instance.

    • Lammy 53 minutes ago
      > I was missing something like a template with the defaults that come with UFW, for instance.

      FreeBSD does include this! It's implemented using IPFW instead of PF. Check out `firewall_type` key in `rc.conf`: https://cgit.freebsd.org/src/tree/libexec/rc/rc.conf?id=8e08...

      For a very easy single-machine firewall, one could set `firewall_type=client` or `firewall_type=workstation` if you want to host anything. For the latter, `firewall_myservices` and `firewall_allowservices` control what ports are enabled and who (other networks/IPs) have access to them.

      For a very simple NAT gateway, one could set `firewall_type=simple` and then `firewall_simple_(iif|inet|oif|onet)(_ipv6)?` to configure the ISP-side and internal-side interface names and IPv4 and IPv6 network ranges for each.

      For more details and to see exactly what each option actually does, check out `/etc/rc.firewall` where this is all implemented: https://cgit.freebsd.org/src/tree/libexec/rc/rc.firewall?id=...

    • roblh 33 minutes ago
      pm2 has been buggy every time I’ve used it, no matter the OS. Incredibly convenient to begin with but simultaneously unpleasant to use software. Updating environment variables with a deployment has not once ever worked as intended.
    • skydhash 58 minutes ago
      > - PM2 was buggy on FreeBSD, which I used to manage my processes

      For supervision?

      > - An alternative, using `rc.d` to run daemons was just so hard to get logs working.

      The unix way is to use logger(1) If you only want some simple message, or redirect to files using newsyslog(8) for managing the sizes of the files.

      > The firewall required too much self configuration to get it right with all the best security practices (ie. What does one do with ICMP.) I was missing something like a template with the defaults that come with UFW, for instance.

      I would recommend The Book of PF[0]. While FreeBSD has syntax difference with OpenBSD's pf, this should give you enough insight on how a firewall operates to get a sense of what rules to write.

      [0]: https://nostarch.com/book-of-pf-4e

  • andix 2 hours ago
    Slightly off topic: What's currently the free Linux distribution with the longest support cycle?

    For a while I used CentOS 7 on all of those small VMs, because it got security updates for a really long time. With minimal risk of breaking things on updates.

    PS: after a bit of research Alma/Rocky Linux are probably the best choices for now. 10 years of support. But are they maintained well?

    • ezst 32 minutes ago
      For a while (a decade+), I was running CentOS on my servers on the same assumption of long time stability and ensuing peace of mind. Then I figured that over such durations, the ecosystem drift becomes significant and keeping applications up to date and running on top of the OS becomes an increasing challenge (with the more "infrastructure" packages like glibc, python/Apache combos, GCC, ... slowly becoming incompatible with the latest applicative stack).

      Then I figured that version upgrades were miserable, not just because I had painted myself in a weird corner with ungodly packages mix-ups, but because the upgrade path was always best-effort. I think I gave up during the 6 to 7 transition, as I realised that all I needed was fedora: with yearly or half-yearly updates I have no need to fight the distro's packages: stuff stays current and in working order, major distro upgrades go smoothly, downtime is minimal. I'm not considering going back to any "server distribution" ever.

    • mhitza 1 hour ago
      > But are they maintained well?

      Alma has a few affordances as it's no longer RHEL source compatible, which means it could ship priviledge escalation fixes with new kernel updates faster.

      Rocky responded with an extra, optional to enable, security repo to provide mitigations to the exploits while waiting for RHEL to downstream.

      Look pretty well maintained to me. If only judging by recent events.

      • gh02t 1 hour ago
        Rocky's docs are also really nice. They aren't as thorough as RedHat's, but they're much more readable and concise, and tend to be written for a less enterprise-y audience.
        • mhitza 1 hour ago
          Don't even remind me about the RedHat docs, lol. Their solutions pages used to be readable with an account, now I think you need a subscription too.

          The manuals, indeed are good, though for more esoteric issues I land too often on a gated answer page.

          • doubled112 39 minutes ago
            You can use the free developer subscription for documentation even if you don't plan to use your 16 RHEL licenses.
      • andix 1 hour ago
        Thanks!

        I don't care much about being fully RHEL compatible, or no ABI changes at all. I just want a system that gets security fixes quickly with as little chances of breaking things as possible.

    • BadBadJellyBean 1 hour ago
      You are betting that whatever you host doesn't live as long as the upgrade cycle because it'll probably be a pain when the upgrades finally arrive. I'd rather have smaller version jumps more often than a huge jump with everything changing after a long time.
      • andix 1 hour ago
        It usually doesn't live until the end of the support cycle. And if it does I will probably migrate it to a fresh VM instead of upgrading the distribution.
    • nextos 46 minutes ago
      I would say NixOS, where it is trivial to switch across releases, run software from different releases, and perform rollbacks.

      I have been running NixOS on several servers for more than a decade. No reinstalling, upgrading, or any breaks whatsoever.

      • indemnity 14 minutes ago
        I run nixOS as well on my home infrastructure (gateway/firewall, a couple of internal servers).

        But I have had, uh, non-trivial breakages happen also when I upgrade the system itself to the next yearly release. Non-bootable kernel kind of breakages.

        But I will give you that I can just boot from the generation before the upgrade, and it works again. So there's that :)

    • secabeen 2 hours ago
      Alma and Rocky if you want fully free or have a lot of machines. RHEL if you are okay with registering with them; they give ten machines free access to their updates for each Registered account in their system.

      RHEL is definitely the most stable major distribution. Alma and Rocky are essentially downstream clones of RHEL.

    • tannhaeuser 1 hour ago
      Debian LTS/extended LTS
      • andix 59 minutes ago
        5 years is not a lot. It releases every 2 years, so it requires upgrading at least every 4 years. In the worst case it's just 3 years of support, if you install right before the next release.

        ELTS is 10 years and paid. It's great that it exists, but not relevant for my toy projects.

        • interroboink 23 minutes ago
          I feel there is a balance to be struck between a project that is popular (where if you run into problems, you will get good support), and one that technically gives longer-term support (but if things go wrong, that support might not be very good).

          I haven't used a lot of different distros, but for me, Debian has been a good balance of those factors. You may need to do more upgrades per decade, but the ones that you do are more liable to go smoothly.

          Just my 2¢ on the topic (:

        • WJW 32 minutes ago
          So there is a project that you care enough about to keep it alive, but 1-2 hours every FOUR YEARS is too much? At some point I just have to call you lazy dude.

          Either the 1-2 hours is a drop in the bucket compared to what you spend on it anyway (like a blog you still regularly update), or you don't actively update the project but still care enough about it to spend half an evening every few years, or you should just admit you don't care about it enough anymore to do even that. In the last case just delete the project.

          • cocoto 24 minutes ago
            It can be way more than 2 hours depending on the project.
          • andix 29 minutes ago
            Yes, I'm lazy. And that's fine.
    • KennyBlanken 2 hours ago
      Probably Debian or Ubuntu. The question is...why do you care that much?

      I've upgraded Debian stable (both pure and with some cherry-picked backports) and Ubuntu (non-LTS and LTS) systems in place and rarely broken anything, for years and years. When stuff has broken it's been a quick google and then slapping myself for not having read the upgrade guide.

      I do generally wait about 2-3 weeks before upgrading, giving time for them to catch stuff that was missed until the great masses were set loose on it.

      • Enginerrrd 44 minutes ago
        Ive had nothing but issues doing that. I think I’ve had a Debian upgrade actually succeed maybe one time? (After some manual intervention to fix some issue other booting on my work server)

        For updates, Debian and Ubuntu are great. For upgrades… not so much for me.

      • andix 2 hours ago
        > why do you care that much?

        I've had issues with Ubuntu/Debian upgrades more than once. Some third party binaries breaking with the update. Or some specific config tweaks that break, because the structure of /etc changed too much.

        For some small VM with a specific purpose I prefer a distribution that changes as little as possible for as long as possible. Less work, more uptime.

        • irishcoffee 1 hour ago
          I won't touch ubuntu unless forced to by some obscure work requirement. I've had enough bad experiences with repos being shut down, updates/upgrades breaking unanticipated, obscure things, and I hate snap.

          The naming conventions drive me crazy as well. When you deal with 2 things that have dumbshit naming conventions, like ubuntu and ROS, its really obnoxious to pretend to case enough to keep track of.

      • nightfly 2 hours ago
        > The question is...why do you care that much?

        Not the OP, but I support Ubuntu as desktop and server OS for an engineering collage and have for 10ish years. Some LTS upgrades don't require many changes (mostly minor package name changes) and some take months of work to get rolled out (mostly for workstations, the server upgrades are usually quick.). Not everything gets upgraded every new OS release. If we had to upgrade everything every 6-12 months it would eat up a significant amount of time for our small team.

        • otherme123 1 hour ago
          I have a machine that has been Fedora since twenty-something to current 44, and upgrading yearly is a breeze. Three commands, and just wait for a download and the reboot. The only thing that breaks if you forget that the upgrade needs attention is the system Postgres, until I migrated to Podman images.
      • secondcoming 1 hour ago
        I had unattended-upgrades cripple our VMs
    • pull_my_finger 8 minutes ago
      [dead]
    • pm2222 1 hour ago
      Use a rolling release like Arch and it’s supported forever.
      • andix 1 hour ago
        I need to enable automatic updates, because I don't have the time to manually update. I have a few machines on Open SuSE Tubleweed, and stuff just randomly breaks. A few months ago there was a weird Kernel bug that just froze all of them. They update and reboot every day, and suddenly it all worked well again. A bit too exciting for me :)
  • rob 2 hours ago
    I've switched to Debian (and since Ubuntu) for my server needs but I remember being obsessed in the mid 2000s with FreeBSD when I was younger. I would spend more time configuring and setting them up than doing anything actually useful on them.

    It used to be hard to find dedicated servers or VPSs with any of the BSDs, I think I settled on Panix.com or something?

    Before that I remember some company called 15MinuteServers (NAC?) out of NJ I think that offered them. Just kind of rambling down memory lane at this point though.

    • indigodaddy 41 minutes ago
      OVH has FreeBSD templates. And most KVM VM/VPS providers will allow console access and mount custom ISO to install whatever you want.
  • Thaxll 2 hours ago
    The benchmarks are completely off, and a recent version of Ubuntu with sane config would easily beat Freebsd.
    • vsgherzi 2 hours ago
      My understanding is that the kernels are mostly equal. I’d be pretty surprised if one had a large impact one way or the other. Any differences I’d chalk up to the userspace program running it.
  • kylec 1 hour ago
    I, too, have a server running 16.04 that I'm afraid to update. It currently has an uptime of 1281 days... at this point I'd feel bad rebooting it
    • hdgvhicv 1 hour ago
      My oldest server is on 8.04.
  • lnenad 2 hours ago
    I love people that aren't afraid to experiment and learn. As someone that hasn't had a formal education in software engineering (just in other kind of engineering) I learned the most by doing and failing.
  • bitbasher 1 hour ago
    I recently switched from Debian based servers to OpenBSD and I have never been happier. I wish I would have done it much, much earlier.
    • pull_my_finger 6 minutes ago
      Can you detail the transition? What were the pain points? I feel like you lose a lot of the selling point of OpenBSD as soon as you start pulling from ports, but how could you do anything productive without it
  • LoganDark 2 hours ago
    > I don’t know why fastfetch always report more memory being used than the actual values. I’ve never seen more than 3GiB used in btop for this server

    My guess would be that fastfetch probably reports actual memory usage while btop probably reports the total usage of all processes. The former is probably higher because of things like filesystem caching

  • not_kurt_godel 1 hour ago
    Boggles my mind that people pay money to host hugo static sites on a VPS, which is objectively inferior and harder in every meaningful way compared to hosting for free on GitHub pages or S3+CloudFront.
    • temp0826 27 minutes ago
      I did it this way for a long time, but it was mostly a learning/experimenting/fun thing back to have a "proper" server out there that I can run whatever service I wanted on (be it an irc bouncer or whatever). I'm a grownup now and don't have the time/care anymore and just run them out of s3/cloudflare (which was still "fun", but now I don't need to worry about the cve of the day. I don't mind contributing to the centralization of the internet when I'm paying $0/month for pages that nobody visits. Definitely happy to nerd out again if something ever warrants it).
    • toast0 1 hour ago
      I pay $35/month for a dedicated server for my nothing webpages. One the one hand, I could really host at home for $0/month extra. Or on a VPS for $5/month. I do need my own thing because I run a network testing tool that needs some amount of direct access.

      On the other hand, dedicated servers are more fun, even if the cpus I get for $35/month are ancient. Is it $30/month fun? Probably not, but it's near zero given my situation.

      At least I'm sensible and don't have an actual colo space to visit.

      In the unlikely event I go viral, I'm pretty sure my server can manage serving https at 1gbps, and that's plenty. Maybe TLS is too much though, the cpus are too old for AESNI.

      • not_kurt_godel 21 minutes ago
        Good on ya. Personally I'm happier with my extra ~$6k, sparing of however many hours of pointless maintenance work over the years, and 100% uptime over past/present/future even if I go into a coma.
    • shric 1 hour ago
      I don’t do it myself, but “objectively inferior in every meaningful way” is a bold claim. It might be harder, but we (geeks) love to do things ourselves.

      If someone is willing to use something like Hugo instead of garbage sites like Medium why not use a VPS? For many people working in tech $10/month and free are the same thing.

      • not_kurt_godel 1 hour ago
        Personally I get my geek satisfaction from building systems that are rock-solid and require zero maintenance. Not choking on rare opportunities to go viral should they arise is a nice bonus too.
        • shric 1 hour ago
          Per a comment I made to one of your other replies in this thread, VPS doesn’t exclude this. You can put it behind cloudflare for free.

          And yes you can have preferences to keep things simple while others can make something unnecessarily complex. For personal projects this is fine and part of learning. If you had said “I much prefer… because…” it would have been fine but you said “objectively inferior in every meaningful way” which ignores people’s subjective preference for over engineering hobby stuff for learning.

    • vsgherzi 1 hour ago
      some people (myself included) like hosting their own stack for fun or for learning.

      There's additional concern with tying your work to something like github it makes it more of a pain to pull it off and put it somewhere else.

      I'm not really sure what you mean by objectively inferior. It's trade offs like everything in this field.

      As far as harder, I don't really think the lift for a personal VPS is that high. Again it's a fun hobby project for most. It's fun to run your own stack.

      If you want to opt into the github cloudflare goodness that's fine they're good services but I wouldn't say it's better or degnegrate others for not doing that.

    • jvanderbot 1 hour ago
      s3 + cloudfront takes approximately 2 extra steps every deploy, and about 10 extra steps that are easy to screw up at setup time. It's not a trivial drop-in, but yes, once it's done it's _really_ done.
      • not_kurt_godel 1 hour ago
        You can make it zero deploy steps beyond git push with CodePipeline, and vibecoding makes the annoying config setup trivial if you know like 20% of what you're doing. There is really zero reason to be using a VPS for this unless you hate money, want your site to choke during once-in-lifetime opportunities to go life-changingly viral, and like contributing to the global population malicious botnets.
        • sgc 44 minutes ago
          OMG, not the once in a lifetime viral opportunity!

          You will never win this crusade, because there are too many people here who know from experience a VPS is neither expensive, nor under-performing up to millions of users a day, nor hard.

          • not_kurt_godel 38 minutes ago
            That's fine, it's not really a crusade. Just my opinion about the right infrastructure for the right use case informed by the objective reasons I gave. If doing more work with more headaches for a solution that costs more and performs worse is your jam, then power to you.
        • qmr 31 minutes ago
          This is a broken take for so many reasons. Also service monitoring is a thing.

          https://en.wikipedia.org/wiki/The_Feeling_of_Power

          • not_kurt_godel 11 minutes ago
            I listed my reasons, feel free to provide your own. I've done enough security, ops, and oncall professionally that I have zero desire to do extra in my free time. Power to you if want to do otherwise and/or post references to "deskilling" while advocating an outdated inefficient approach to long-solved problems.
        • shric 1 hour ago
          > want your site to choke during once-in-lifetime opportunities to go life-changingly viral, and like contributing to the global population malicious botnets.

          You can put it behind cloudflare for free.

          • not_kurt_godel 1 hour ago
            You can put the whole thing on CloudFlare Pages for free too. Zero reason to pay for or deal with the complexity of an unnecessary VPS.
            • shric 1 hour ago
              Again “zero reason”. For some people it’s fun to have a VPS, that is a reason in and of itself.
              • not_kurt_godel 57 minutes ago
                Of course literally anything can be 'reasonable' if you a priori like doing it independent of its technical/functional merits. My ex-coworker liked to write literally tens of thousands of lines of extra, completely unnecessary code for fun. That was a fine reason for him personally but didn't make it any less stupid to deal with for the rest of us.
        • vsgherzi 1 hour ago
          This is a blog.... you don't need some monster machine. You can server TONS of people off the smallest Digital Ocean instance.

          Many of these small VPSs can be had for less than a couple bucks a month. Tons of popular influencers run their own machines for their blog.

          insinuating that it's unsafe to run your own machine is insanity. I don't understand this mindset of being scared to run your own stuff. Especially if you're doing doing it at such a large scale there's nothing wrong with doing it with nginx and a linux box on a vps. You'll learn a hell of a lot more and be fine. At the end of the day it's a computer. We've been hosting websites since the 70's. With the advant of cloud compute is easier than every to run your own.

          (edited to be less mean)

          • not_kurt_godel 56 minutes ago
            We have had something vastly better than an individual computer since idk, the mid 90s, called a CDN.

            I guess if you want to call being informed about the online threat landscape "scared", that's your perogative. For me, it's common sense to avoid completely unnecessary threat vectors to my digital infrastructure, but power to you if you like dealing with extra maintenance overhead and constantly wondering whether you're providing free cryptomining to some random international criminal.

            • vsgherzi 33 minutes ago
              There's threats on the internet, so don't spin up servers? Idk am I reading into that unfairly? That seems pretty fear mongering to me. Lots of engineering goes into making things safe for engineers to build on. Of course you can also just use squarespace and not worry about it at all. Perhaps my security posture is just not as intense as yours but I'm really just not super concerned my blog is going to get pwned. If it does then I get to learn some interesting things.

              I'm also not sure that I really need a CDN for a simple blog . I'm not going to benefit from the caching as it's not video or images.

              • not_kurt_godel 7 minutes ago
                Servers are work, including security overhead, so yes, don't spin them up if there is an alternative solution that is superior in every way except for not being able to churn digital butter.
    • dizhn 1 hour ago
      Another option is cloudflare pages. Can be coupled with any hub or you can just push html artifacts.
  • waynesonfire 1 hour ago
    I was running Ubuntu 16.04; migrated to FreeBSD and I'm all in. Between 16.04 and the current version of Linux; the ecosystem shifted. It's values shifted in ways that did not align with me. This mis-alignment is what motivated me to boot-up FreeBSD. I'm glad I discovered it. I found my happy place again.

    It's an incredible journey to take--whether you stick with it or not. Migrating to FreeBSD gives you new eyes into what Linux was, is, and the awesomeness of FreeBSD that is so hard to articulate; like describing the color blue. It must be taken as a whole to appreciate it; and I'm not just saying the OS, it's commands, kernel features, but, the end-to-end compute experience, over time.

    If I could draw an equivelent, it would be like when Djistrka savagely destroyed the GOTO statement with a single, short, paper. It took a brilliant mind to articulate that and there has yet to be such a mind to describe the beauty of FreeBSD. So, the best I can do, is just to challenge you to try it.