Permalink

Spamalamadingdong

Did I just spend an inordinate amount of time tonight making sure my email has its SPF, DKIM, and DMARC configurations are set up properly, even for addresses that only ever send mail to me? Yes, yes I did. While mine had been set up correctly already, I read the other day about upcoming / already implemented email provider changes and figured why not make my settings stricter now too? All it took was changing “?all” to “-all” in my SPF configuration. The part that confused me the most after that was that even though I’m sending emails through Fastmail, if I sent mail directly through their webmail or via my desktop mail client everything would pass all checks fine, but when I’d send emails from my applications or other systems they’d fail with error messages like:

(glennfitzpatrick.com: Sender is not authorized by default to use 'foo@glennfitzpatrick.com' in 'mfrom' identity (mechanism '-all' matched))

I was really scratching my head since they were all using the same credentials, and I couldn’t figure out a way to adjust the SPF DNS setting to permit specific email addresses.

After some research, it turns out that if you don’t have the email address you’re sending as added to your account (even just as an alias is fine), then when it’s sent it goes through their forwarding server which causes SPF to fail. Once I made sure all my application-specific emails were added to my account as aliases my SPF checks were passing once again but now in strict mode.

In other news, today’s weather was nice, and the Capitals are going to the playoffs!

Permalink

You could say they were “shoeperlative”

I bought some new shoes today. Last week when Katie and I went to upstate New York to see the eclipse we did some sightseeing to see the waterfalls around Ithaca, and I learned two things:

  1. I should have brought/worn hiking boots, and
  2. It was time to get new shoes.

It’s not that my shoes were bad or uncomfortable. It’s that when hiking on a trail with plenty of mud, if the shoes that you’re wearing don’t really have any tread on them, you’re going to go slipping and sliding and it’s a wonder that I didn’t splat. I say we were “hiking” on a “trail” but it was more like walking on a wide, flat path where if it had been a bit drier there wouldn’t have been any mud. But mud there was, and slip-sliding I went in my casual sneakers that look more like running shoes that I picked up on a whim at Costco to replace my old casual gray canvas Nike shoes after they got a bit old and stained and floppy and worn. Hiking boots would have been a better choice (both for the ruggedness as well as the activity) but for some reason it didn’t occur to me to pack them.

Later on the same trip I went walking across what I expected to be a dry field, but soon I started squelching in the grass. I tried to power through but it just got squelchier and squelchier so I jumped up on some nearby rocks to avoid the bog I found myself in, but when I thought I had safely circumnavigated the swamp and jumped back off the rocks I landed right in the thick of it and spent the day waiting for the eclipse with muddy shoes and soggy socks. When I got home from the trip I tossed my shoes in the wash and while they’re now clean, it was time to start thinking about replacing them.

So today I went out looking for new shoes. First stop was Dick’s Sporting Goods and while they had some casual-looking shoes that I could have bought if I needed a new pair right then and there, since none of the shoes really spoke to me (despite having tongues, wakkawakka) and I wasn’t in dire straits shoe-wise I continued with my errands.

Later I wandered in to a Rack Room Shoes and wouldn’t you know I found the exact same pair of casual grey canvas Nike shoes that I had just before my current pair? I was surprised to find them so easily as when I had looked at Zappos’ site the other day I couldn’t find any of my old shoe styles available, or if they were still made they didn’t have them in my size, but finding the old shoe style I really liked available in my size in the second store was not something I was expecting.

Permalink

~*~ away messages are no longer a thing when folks are perpetually online ~*~

I was cleaning up some files on my computer and found I had a directory of old iChat / AIM chat conversation logs going back to 2003 still floating around. Unfortunately they couldn’t be opened directly with the Messages app now on my Mac, and were some sort of binary file that needed to be decoded so I couldn’t simply open them up with a text editor to view the messages themselves. Luckily I found an application called Past for iChat that let me not only open up my old chats, but also let me convert them to Markdown and HTML and PDF files, so I’ve now got over 6,000 conversations from 2003–2011 to look through. I’ve already skimmed through some and found some pretty funny bits from ol’ college Glenn, and many more exchanges that had me stifling laughs, but for every great quote there’s many more that are just “hehehe” or “lolol”.

Now to figure out how I’d like to save some of my favorite quotes…

Permalink

Kerouac and Kirk

The other day I went down a rabbit hole of reading through some old LiveJournal posts written by a romantic interest from more than half a lifetime ago (he says as he crumbles into dust) and it reminded me how when you’re 19 years old you’d put just anything out there in your LiveJournal for public consumption. On one hand, yikes, 19-year-olds would put anything and everything out on the internet back in the day, but on the other it was fascinating to revisit all of the tech issues of the early 2000s like phone lines and AOL and Geocities and cellular phones having unlimited night and weekend minutes and to read what songs on a CD made a long-distance college girlfriend think of me.

I recently updated my installation of iA Writer and saw they’ve now got Apple Shortcuts functionality included, with one of the example Shortcuts being a way to automatically open up a text file saved dated with the day’s date and start a new line with the current time, all ready for typing out a diary entry. I thought that was pretty handy as a way to keep a local, more private diary. Not that this blog will be going away, but some things I’d want to type up for my own reference for the future and not necessarily for public consumption, and I had been trying to figure out a good way to separate the more private diary writing from the blog-style journal entries intended for public consumption. I had experimented previously with Day One, but what I really wanted was just something like a single text file I’d continuously append to.

Permalink

Mastodon on Mac

I’ve been playing off and on with trying to get Mastodon running on my self-hosted Mac mini colocated at MacStadium (yes, it’s a referral link, but I’m just a really happy customer!) and with the release of Mastodon 4.2.0 I think I’ve finally got it up and working! I’ve heard of a few other people mention getting it to work on their Macs but I haven’t yet seen any real how-tos on, uh, “how-to” do it, so I figured I’d write up what I needed to do to get it working and document the pitfalls and caveats that I encountered along the way. While I could have hosted it on AWS or DigitalOcean or used one of the Mastodon hosting services, I was already paying for my Mac mini server and it had more than enough power to host a small Mastodon instance.

I’ve created a Github repository to keep track of any future changes and updates if necessary.

Important notes

Because I was already using my Mac mini to host my blog as well as some other applications, I was already running the following applications and services, so a few of these might need to be installed but I already had them in place, and the list of prerequsites might thus be incomplete:

  • Homebrew
  • Apache httpd
  • MariaDB
  • Redis
  • ffmpeg
  • ImageMagick
  • Supervisor

Almost all of the information I saw about configuring the webserver for Mastodon was using Nginx, and rather than run a second webserver I took the example Nginx configuration file and rewrote it for httpd as well as I could. I think I’ve got it all translated appropriately, but there might be some configuration commands I may have neglected or that may be entirely superfluous. Actually, now that I think of it, I guess that disclaimer applies to this whole writeup, heh. But, it seems to work!

Besides my Mac mini at MacStadium, I also use Amazon Web Services for some additional hosting features, and wanted to use S3 for my media uploads and host them through the CloudFront CDN; I didn’t try getting it set up to host my media files locally so I’m not sure what issues there may be with that configuration, and I’m not going to get into the S3 bucket creation and CloudFront configuration steps at this time.

I also use Fastmail (yes, another referral link, but again I’m just really happy with their service!) for my mail hosting, and am using their SMTP server to send emails from the Mastodon server. I couldn’t get it working using STARTTLS, so it’s using the SSL server instead.

Also, when I had once previously tried to get Mastodon working on my Mac I was having all sorts of issues with getting Ruby installed, and I eventually discovered Ruby on Mac which got it installed for me.

Install prerequisites

This is possibly an incomplete list of prerequisites:

brew install postgresql@14
brew install redis
brew install nodenv
brew install yarn
brew install imagemagick
brew install httpd
brew install libidn
brew install opensearch
brew install ffmpeg

Ruby

Because I already had mariadb installed as my MySQL-compatible database, I didn’t need ruby-on-mac to install the official mysql package as part of its web development tool installation, so I commented out brew 'mysql' from Brewfile-rom-webdev before I ran its installation command.

Once ruby-on-mac was installed, make sure we’re using an appropriate version of Ruby.

chruby 3.2.2

NodeJS

nodenv install 20.5.1

This might be able to go earlier in the process, but I wasn’t sure if ruby-on-mac installed anything that NodeJS might need.

Start and set up supporting services

brew services start opensearch
brew services start redis
brew services start postgresql@14

Create the mastodon database user.

psql postgres

CREATE USER mastodon WITH PASSWORD 'password' CREATEDB;
\q

Get Mastodon

cd ~/Sites
git clone https://github.com/mastodon/mastodon.git
cd mastodon
git checkout v4.2.0

Compile Mastodon

corepack enable
yarn set version classic
gem install bundler --no-document
bundle config deployment 'true'
bundle config without 'development test'
bundle install -j$(getconf _NPROCESSORS_ONLN)
yarn install --pure-lockfile

Configure Mastodon

DISABLE_DATABASE_ENVIRONMENT_CHECK=1 needed if rebuilding over an existing database.

NODE_OPTIONS=--openssl-legacy-provider is required to compile assets.

You may not be able to send a test email during the configuration process until some additional values are added to the .env.production file in the next section.

RAILS_ENV=production DISABLE_DATABASE_ENVIRONMENT_CHECK=1 NODE_OPTIONS=--openssl-legacy-provider bundle exec rake mastodon:setup

Configure .env.production file

You should now have an .env.production file that looks something like this, but showing the values you entered during the configuration phase:

LOCAL_DOMAIN=example.com
SINGLE_USER_MODE=false
SECRET_KEY_BASE=
OTP_SECRET=
VAPID_PRIVATE_KEY=
VAPID_PUBLIC_KEY=
DB_HOST=localhost
DB_PORT=5432
DB_NAME=mastodon_production
DB_USER=mastodon
DB_PASS=
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_PASSWORD=
S3_ENABLED=true
S3_PROTOCOL=https
S3_BUCKET=
S3_REGION=us-east-1
S3_HOSTNAME=s3.us-east-1.amazonaws.com
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
S3_ALIAS_HOST=
SMTP_SERVER=
SMTP_PORT=465
SMTP_LOGIN=
SMTP_PASSWORD=
SMTP_AUTH_METHOD=plain
SMTP_OPENSSL_VERIFY_MODE=none
SMTP_ENABLE_STARTTLS=never
SMTP_FROM_ADDRESS=

Now I needed to add a few additional environment variables to the file. Remember what I said earlier about not being able to send emails during the configuration phase? It turned out I needed to add the following lines to the .env.production file in order to connect to the SMTP server:

SMTP_TLS=false
SMTP_SSL=true
SMTP_CA_FILE=/opt/homebrew/etc/openssl@3/cert.pem

I already had my site running on my main domain and wanted to host Mastodon on a subdomain:

WEB_DOMAIN=social.example.com

I had issues with images and other assets not loading properly, so I had to have Rails serve them:

RAILS_SERVE_STATIC_FILES=true

I also enabled Elasticsearch (well, OpenSearch):

ES_ENABLED=true
ES_HOST=localhost
ES_PORT=9200

Create the Elasticsearch indices

I had to first comment out the progress.total = indices.sum { |index| importers[index].estimate! } line due to a progress bar error (see #18625).

RAILS_ENV=production bin/tootctl search deploy

Deploy the mastodon_supervisor.ini configuration file

I used an example file from checkmyworking.com. You may need to create the /opt/homebrew/etc/supervisor.d directory, but this file goes there; replace foo with your username and update any paths to point to where you cloned the mastodon directory earlier if needed.

Note that PGGSSENCMODE=disable is needed due to a segmentation fault in gem pg on MacOS.

Update httpd configuration files

Because I wanted to host my server in a subdomain, but didn’t want that subdomain to be part of the name of my instance, I had to forward some URIs from the main domain to the Mastodon subdomain; these lines were added in my VirtualHost section for my main domain:

<VirtualHost xxx.xxx.xxx.xxx:443>
  ServerName example.com
  ...

  <Location "/.well-known">
    ProxyPreserveHost On
    RequestHeader set X-Real-IP %{REMOTE_ADDR}s
    RequestHeader set X-Forwarded-Proto "https"
    ProxyPass http://127.0.0.1:3000/.well-known
    ProxyPassReverse http://127.0.0.1:3000/.well-known
  </Location>

  <Location "/api">
    ProxyPreserveHost On
    RequestHeader set X-Real-IP %{REMOTE_ADDR}s
    RequestHeader set X-Forwarded-Proto "https"
    ProxyPass http://127.0.0.1:3000/api
    ProxyPassReverse http://127.0.0.1:3000/api
  </Location>

  <Location "/oauth/token">
    ProxyPreserveHost On
    RequestHeader set X-Real-IP %{REMOTE_ADDR}s
    RequestHeader set X-Forwarded-Proto "https"
    ProxyPass http://127.0.0.1:3000/oauth/token
    ProxyPassReverse http://127.0.0.1:3000/oauth/token
  </Location>

  <Location "/oauth/authorize">
    RewriteEngine On
    RewriteRule .* https://social.glennfitzpatrick.com%{REQUEST_URI} [R=301,L]
    Header append Access-Control-Allow-Origin '*'
  </Location>

  ...
</VirtualHost>

Meanwhile, the social.example.com subdomain that would actually host the web interface, its configuration file is here. I commented out configuration details that were in the official nginx configuration file that I either couldn’t find an corresponding configuration command for in httpd, or they were configuration details that didn’t appear to be needed. Update the domain name, certificate file locations, and Mastodon repository location and deploy it to wherever you keep your httpd config files; my Homebrew installation has them in /opt/homebrew/etc/httpd/sites.

Database tweaks

I updated the /opt/homebrew/var/postgresql@14/postgresql.conf file to tune the database server to better suit my hardware using PGTune (I could probably increase the total RAM, but I figured I’d start with this setup for now and see how it runs):

# DB Version: 14
# OS Type: mac
# DB Type: web
# Total Memory (RAM): 4 GB
# CPUs num: 8
# Data Storage: ssd

max_connections = 200
shared_buffers = 1GB
effective_cache_size = 3GB
maintenance_work_mem = 256MB
checkpoint_completion_target = 0.9
wal_buffers = 16MB
default_statistics_target = 100
random_page_cost = 1.1
work_mem = 1310kB
huge_pages = off
min_wal_size = 1GB
max_wal_size = 4GB
max_worker_processes = 8
max_parallel_workers_per_gather = 4
max_parallel_workers = 8
max_parallel_maintenance_workers = 4

The PgHero tool in the Mastodon server also prompted me to make a change to enable statistics, so these lines were also added to the postgresql.conf file:

shared_preload_libraries = 'pg_stat_statements'
pg_stat_statements.track = all

Then enable the statistics for the mastodon database user:

psql postgres

\c mastodon;
CREATE extension pg_stat_statements;
exit

Start Mastodon!

Whew, I think that’s about everything that I encountered while getting it to run properly. Now you should be able to start Mastodon directly in two Terminal windows…

RAILS_ENV=production PORT=3000 MAX_THREADS=10 WEB_CONCURRENCY=4 PGGSSENCMODE=disable bundle exec puma -C config/puma.rb
RAILS_ENV=production DB_POOL=25 MALLOC_ARENA_MAX=2 LD_PRELOAD=libjemalloc.so PGGSSENCMODE=disable bundle exec sidekiq -c 25

Or, as I prefer, with supervisor:

brew services start supervisor

Now you should be able to use Ivory or your Mastodon client of choice to connect to your example.com instance, and access it on the web at social.example.com! The only issue I can seem to find at the moment is that when I list another account on my same instance on my profile, if I try to access that other account in my client I get a “User not found” error but I can’t figure out if it’s something with the API URLs or something with the client or what, as it seems to think the instance should be social.example.com. But other than that, things seem to work well! Hope this helps!