I spent three hours last week debugging a GitHub Actions workflow that kept failing silently. You know the drill—digging through YAML syntax, checking environment variables, cursing at indentation. Then I tried something different: I pasted the broken workflow into Claude Code and said “This GitHub Action isn’t working, fix the deployment script and explain what was wrong.” Two minutes later, I had a working solution and learned that my Docker context was misconfigured.

The phrase “code is cheap” has never been more accurate, but there’s a twist: the real value now lies in knowing how to communicate with AI. As AI becomes increasingly sophisticated, the bottleneck is no longer writing code—it’s crafting the right prompts to get AI to write it for you.

This isn’t just about faster development—it’s about fundamentally different skills. The developers who are thriving now aren’t necessarily the ones who can write the most elegant algorithms or memorize framework APIs. They’re the ones who can break down complex problems into clear, specific instructions that AI can understand and execute. It’s like being a really good technical writer, except your audience is a machine that can code better than most humans but needs explicit guidance on what to build.

Core Prompting Skills

Learning to work with AI isn’t rocket science, but it does require unlearning some bad habits. Most developers are used to diving straight into implementation details. With AI, you need to step back and think like a project manager first—define the what, why, and how before asking for code.

Context is everything. The difference between “create a login form” and “create a login form for this Next.js TypeScript project using our existing shadcn/ui components, integrate with our Supabase auth, and match the design system in components/ui/” is the difference between generic boilerplate and code that actually fits your project. I learned this the hard way after getting beautiful React components that used completely different styling libraries than the rest of my app.

Specificity saves time. Vague requests lead to back-and-forth iterations that could have been avoided. Instead of “make this faster”, try “optimize this database query that’s taking 3+ seconds on tables with 100k+ rows, focus on indexing and avoiding N+1 queries.” The AI can’t read your mind, but it can work miracles when you’re explicit about constraints, performance requirements, and edge cases you’re worried about.

Think in steps, not solutions. When tackling complex problems, resist the urge to ask for everything at once. Break it down: “First, analyze this messy legacy code and identify the main architectural issues. Then suggest three refactoring approaches with pros and cons. Finally, implement the safest approach with proper error handling and tests.” This step-by-step approach not only gets better results but also helps you understand the AI’s reasoning process.

Ask for the plan before the code. Before diving into implementation, get the AI to think through the architecture first. Try “Plan out how to add real-time notifications to this app - what components need to change, what new APIs are required, and how should we handle state management?” Once you approve the approach, then ask for the actual code. This prevents those moments where you realize the AI built something elegant but completely wrong for your use case.

Prompt Templates That Actually Work

After months of trial and error, I’ve collected a set of prompt patterns that consistently produce good results. Think of these as your starter templates—modify them for your specific context, but the structure works across different tools and scenarios.

Architecture Decisions

“Design a [system/component] for [specific use case] that needs to handle [requirements like scale, performance, security]. Current tech stack is [languages/frameworks]. Consider [specific constraints like budget, timeline, team skills]. Provide 2-3 approaches with trade-offs and recommend the best option with reasoning.”

This template works because it forces you to define the problem scope, constraints, and decision criteria upfront. I used this recently for designing a caching layer and got three solid approaches I hadn’t considered, complete with implementation complexity analysis.

Code Review

“Review this [language] code for [specific concerns like security vulnerabilities, performance bottlenecks, maintainability issues]. Focus on [areas of concern]. For each issue found, provide the problematic code snippet, explain why it’s problematic, and show the improved version with explanation.”

The key here is being specific about what you’re worried about. Generic “review this code” gets generic feedback. But asking for security review of authentication logic gets you detailed analysis of timing attacks, input validation, and session management issues you might miss.

Refactoring

“Modernize this [legacy technology/old pattern] code to use [new framework/modern patterns]. Maintain the same functionality and API contracts. Improve [specific aspects like type safety, error handling, performance]. Show the transformation step-by-step and explain each change.”

I am using this template to migrate my Jekyll blog components to Next.js, converting Liquid templates to JSX components while preserving the same styling and functionality. The “step-by-step” part is crucial—it helps you understand the changes and makes the refactoring easier to review and test.

Test Generation

“Generate comprehensive tests for this [component/function/API] covering happy path, edge cases, and error conditions. Use [testing framework like Jest, pytest, RSpec]. Include setup/teardown code, mock external dependencies, and test for [specific scenarios like race conditions, invalid inputs, network failures]. Aim for [coverage percentage] code coverage.”

This template has saved me countless hours of thinking through test scenarios. I recently used it for testing a payment processing function and got tests for edge cases I’d never considered—like what happens when the payment gateway returns a success response but the database write fails.

Debugging/Troubleshooting

“This [language/framework] code is throwing [specific error]. Here’s the error message: [paste error]. Here’s the relevant code: [paste code]. Help me identify the root cause and provide a fix with explanation of why this happened.”

The magic is in providing both the error message and the actual code. I’ve wasted too many hours getting generic debugging advice because I only shared the error message. When you include the problematic code, AI can spot issues like variable scope problems or async/await misuse that the error message doesn’t make obvious.

Performance Optimization

“This [function/query/component] is performing slowly under [specific conditions]. Current performance is [metrics]. Analyze the bottlenecks and suggest optimizations. Focus on [specific areas like database queries, memory usage, rendering].”

Including actual performance metrics makes all the difference. Instead of getting generic advice like “use indexes,” you get targeted solutions. When I mentioned my ML model inference was taking 800ms per request, I got specific suggestions about batch processing and model quantization that cut the response time to under 200ms.

Documentation Generation

“Generate comprehensive documentation for this [codebase/module/API]. Include overview, installation steps, usage examples, API reference, and common troubleshooting. Target audience is [developers/end-users]. Follow [documentation style like JSDoc, Sphinx].”

This template works because it defines the scope and audience upfront. Documentation for end-users is completely different from developer docs. I’ve used this to generate API documentation that was actually usable, not just a wall of technical jargon.

API Integration

“Help me integrate with [API name] to [specific goal]. I need to [specific actions like authenticate, fetch data, handle webhooks]. Handle rate limiting, error responses, and provide proper TypeScript types. Use [HTTP library].”

The key is being explicit about error handling and types. Generic API integration examples rarely handle real-world issues like rate limiting or webhook verification. This template gets you production-ready code that won’t break when the API returns unexpected responses.

The Shift is Real

Code is getting cheaper to produce, but knowing how to communicate with AI is becoming more valuable. Master the prompting skills, try the templates, and see what works for your workflow. The tools will keep evolving, but the fundamentals of clear communication with AI will remain.

Start small, be specific, and remember: the best prompt is the one that gets you exactly what you need.

⚠️ Usage Warning

This technique should only be used for legitimate aggregation of publicly available RSS content. Do not use it for bypassing paywalls, accessing private content, or commercial redistribution.

When setting up my Glance dashboard to display feeds from various content platforms, I ran into a frustrating problem: images weren’t loading. Instead of the expected thumbnails and cover images that make a dashboard visually engaging, I was seeing broken image placeholders scattered throughout my feeds.

The Problem

Opening the browser’s developer console revealed the culprit immediately. Error messages like this were appearing repeatedly:

GET https://cdn.example.com/images/photo.jpg 403 (Forbidden)
Failed to load resource: the server responded with a status of 403 (Forbidden)

These 403 Forbidden errors were blocking external images from RSS feeds, yet the same URLs worked when opened directly in the browser. The image servers’ referrer policy of strict-origin-when-cross-origin was causing them to reject requests coming from my domain, triggering anti-hotlinking protection.

Understanding the Root Cause

The problem stemmed from how content platforms protect their images from unauthorized use. Most platforms implement anti-hotlinking protection by checking the Referer header in image requests. Here’s what was happening:

1. Referrer checking: When my dashboard requested an image, the CDN server would examine the Referer header.
2. Domain mismatch: Since requests came from my dashboard’s domain rather than the platform’s own website, the server treated them as unauthorized.
3. Inconsistent blocking: Different CDN subdomains had varying levels of protection, explaining why some images loaded while others didn’t.

This was particularly maddening because the same images would load perfectly when I opened them directly in a new browser tab, but they’d fail when embedded in my dashboard. The RSS feeds themselves parsed correctly with all the text content intact, but without the visual elements, my dashboard looked broken.

Cloudflare Workers: From Perfect Solution to Dead End

My first instinct was to reach for Cloudflare Workers—it used to be the perfect solution for this kind of proxy problem. I could write a simple function to intercept image requests, fetch them with the proper headers, and return them with CORS headers enabled. The edge computing model would even provide great performance with global distribution.

However, when I checked Cloudflare’s current Terms of Service (last updated December 3, 2024), I discovered a clause in section 2.2.1 that stopped me in my tracks:

“(j) use the Services to provide a virtual private network or other similar proxy services.”

This restriction effectively rules out using Workers for image proxying, as it would fall under “similar proxy services.” While this clause may have existed before, Cloudflare has been increasingly strict about enforcing it.

The risk simply wasn’t worth it. Getting my account suspended would affect not just this dashboard project, but potentially other services I was running on the Cloudflare. I needed a solution that was both effective and compliant with current service terms.

Self-Hosted Nginx Reverse Proxy: The Solution

Since I was already running Glance on my own VPS, I decided to implement a reverse proxy using Nginx on the same server. This approach would give me complete control while avoiding any third-party terms of service violations. The beauty of this solution is that it uses my existing infrastructure without requiring additional hosting costs.

Setting Up Server-Side Caching

I added a server-side cache to the Nginx configuration to improve performance and reduce load on the external CDNs. This cache stores images for 14 days (336 hours), ensuring that frequently viewed images load instantly on subsequent visits.

# /etc/nginx/nginx.conf
http {
    ##
    # Cache Settings
    ##
    proxy_cache_path /var/cache/nginx/images
                     levels=1:2
                     keys_zone=images:10m
                     max_size=1g
                     inactive=336h
                     use_temp_path=off;

    # ... rest of http config
    include /etc/nginx/sites-enabled/*;
}

Configuring the Image Proxy

Then I configured the image proxy in my site configuration. The key insight was using dynamic DNS resolution to handle multiple CDN providers through a single proxy path. Here’s the configuration I added to my existing server block:

# /etc/nginx/sites-available/dashboard.example.com
server {
    server_name dashboard.example.com;

    # DNS resolver for dynamic proxy
    resolver 127.0.0.53 valid=300s;
    resolver_timeout 5s;

    # Existing Glance application
    location / {
        proxy_pass http://localhost:8080;
        # ... existing headers
    }

    # Image proxy for external CDNs
    location ~ ^/img-proxy/([^/]+)/(.*)$ {
        set $backend_host $1;
        set $backend_path $2;

        # Enable proxy caching
        proxy_cache images;
        proxy_cache_valid 200 336h;
        proxy_cache_key "$backend_host$backend_path";
        proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;

        # Proxy to external CDN
        proxy_pass https://$backend_host/$backend_path;

        # Essential headers for bypassing anti-hotlinking
        proxy_set_header Referer "https://$backend_host/";
        proxy_set_header User-Agent "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36";
        proxy_set_header Host $backend_host;

        # SSL configuration
        proxy_ssl_verify off;
        proxy_ssl_server_name on;

        # Add CORS and cache headers
        add_header Access-Control-Allow-Origin "*" always;
        add_header Cache-Control "public, max-age=1209600" always;
        add_header X-Cache-Status $upstream_cache_status always;

        # Basic protection against abuse
        valid_referers none blocked dashboard.example.com;
        if ($invalid_referer) {
            return 403;
        }
    }

    # ... existing SSL configuration
}

The magic happens in the regex location block. By capturing the hostname and path in separate variables, I could support any external CDN domain through my proxy. The URL transformation is clean:

Original:  https://cdn.platform.com/images/photo.jpg
Proxied:   https://dashboard.example.com/img-proxy/cdn.platform.com/images/photo.jpg

After setting up the configuration, I created the cache directory:

sudo mkdir -p /var/cache/nginx/images
sudo chown -R www-data:www-data /var/cache/nginx/images
sudo chmod -R 755 /var/cache/nginx/images
sudo nginx -t && sudo systemctl reload nginx

Automating Image URL Conversion

With the Nginx proxy in place, I needed a way to automatically convert external image URLs in my Glance dashboard. Rather than manually updating every RSS feed configuration, I wrote a client-side script that would handle the conversion transparently.

I added this JavaScript to my Glance configuration as a custom .js file:

(function () {
  // CDN domains that commonly implement anti-hotlinking
  const BLOCKED_DOMAINS = ["cdn.example.com", "images.platform.com", "static.service.net"];

  function convertExternalImages() {
    const selector = BLOCKED_DOMAINS.map((domain) => `img[src*="${domain}"]:not([data-converted])`).join(",");

    document.querySelectorAll(selector).forEach((img) => {
      try {
        const originalSrc = img.src;
        const proxySrc = originalSrc.replace(/https?:\/\/([^/]+)\//g, "/img-proxy/$1/");

        img.src = proxySrc;
        img.setAttribute("data-converted", "true");
        console.log("Converting:", originalSrc, "→", proxySrc);
      } catch (error) {
        console.error("Failed to convert image:", error);
      }
    });
  }

  if (document.readyState !== "loading") {
    convertExternalImages();
  } else {
    document.addEventListener("DOMContentLoaded", convertExternalImages);
  }

  // Check for new images every 3 seconds
  setInterval(convertExternalImages, 3000);
})();

The script maintains a list of problematic CDN domains and only processes images from those sources. When it detects a blocked image, it uses regex to rebuild the URL to route through my proxy. This focused approach avoids wasting resources on images that already load fine.

Results and Testing

After implementing both components, I tested the setup:

// Test in browser console
fetch("/img-proxy/cdn.platform.com/images/test.jpg")
  .then((response) => console.log("Status:", response.status, response.statusText))
  .catch((error) => console.error("Error:", error));

The test returned Status: 200 OK, confirming the proxy was working correctly with proper CORS headers.

With the proxy confirmed working, the dashboard transformation was impressive. Images that had been broken placeholders now loaded correctly throughout my feeds. RSS content became visually rich with thumbnails and cover images that made the dashboard much more engaging.

The JavaScript automatically converts problematic image URLs as new RSS content loads. When fresh content appears, any images from blocked domains are quickly converted to use the proxy route. The 14-day cache configuration means once an image loads through the proxy, subsequent views are instant.

Verifying Cache Performance

To verify that the caching was working as intended, I checked the Nginx cache directory:

# Count cached files
sudo find /var/cache/nginx/images -type f | wc -l

# Check cache size
sudo du -sh /var/cache/nginx/images

The cache was working as intended, with files being created and deleted as expected.

Responsible Usage

This solution requires thoughtful implementation to respect both technical constraints and content platforms. The 14-day caching configuration significantly reduces load on source CDNs—an image that might be viewed dozens of times only generates a single upstream request.

The referrer validation prevents unauthorized access. The valid_referers directive restricts proxy usage to requests originating from my dashboard domain, ensuring it can’t be exploited as an open relay by external users.

When implementing similar solutions, consider the scope and purpose of your usage to maintain ethical standards and avoid potential service violations.

I recently switched from Docker Desktop to OrbStack because of its better performance and native macOS integration. One of OrbStack’s convenient features is automatic domain name assignment for containers. Instead of remembering port numbers, containers get clean URLs like https://container-name.orb.local with automatic HTTPS certificates. This makes accessing development services much more elegant than dealing with localhost:3000, localhost:4000, etc.

However, when I maintain my Jekyll-based website in a container, this convenient feature introduced an unexpected issue with LiveReload that took some detective work to understand.

When I try to use the OrbStack generated domain name https://jekyll.zhipenghegithubio.orb.local/ to access my Jekyll site, LiveReload crashes immediately with the following error:

jekyll-1  | LiveReload experienced an error. Run with --trace for more information.
jekyll-1  | /usr/local/bundle/gems/jekyll-4.4.1/lib/jekyll/commands/serve/websockets.rb:44:in 'HTTP::Parser#<<': Could not parse data entirely (0 != 247) (HTTP::Parser::Error)
jekyll-1  | 	from /usr/local/bundle/gems/jekyll-4.4.1/lib/jekyll/commands/serve/websockets.rb:44:in 'Jekyll::Commands::Serve::HttpAwareConnection#dispatch'
jekyll-1  | 	from /usr/local/bundle/gems/em-websocket-0.5.3/lib/em-websocket/connection.rb:79:in 'EventMachine::WebSocket::Connection#receive_data'
jekyll-1  | 	from /usr/local/bundle/gems/eventmachine-1.2.7/lib/eventmachine.rb:195:in 'EventMachine.run_machine'
jekyll-1  | 	from /usr/local/bundle/gems/eventmachine-1.2.7/lib/eventmachine.rb:195:in 'EventMachine.run'
jekyll-1  | 	from /usr/local/bundle/gems/jekyll-4.4.1/lib/jekyll/commands/serve/live_reload_reactor.rb:42:in 'block in Jekyll::Commands::Serve::LiveReloadReactor#start'

If revisiting the page, it will show 502 Bad Gateway error and OrbStack proxy error: dial tcp 192.168.97.2:8080: connect: connection refused.

The WebSocket parsing error doesn’t just break LiveReload—it crashes the entire Jekyll server. This means the site becomes completely inaccessible until the container is restarted. Here’s what I discovered after systematically testing different configurations.

Testing with Another WebSocket Application

Facing this crash, I needed to understand what was causing it. Was this an issue with OrbStack’s reverse proxy, Jekyll’s implementation, or something else entirely? To isolate the problem, I decided to test whether other WebSocket applications had the same issue in the identical setup.

docker run -p 8080:8080 jmalloc/echo-server

I then tested both access methods:

• Direct access: http://localhost:8080/.ws
• OrbStack domain: https://condescending_solomon.orb.local/.ws (The container name condescending_solomon is automatically generated)

Result: Both worked perfectly! The WebSocket echo server had no problems connecting and echoing messages through either direct access or OrbStack’s domain routing.

Looking at the browser’s network tab revealed the difference in WebSocket connection types:

• Direct: ws://localhost:8080/ (plain WebSocket)
• OrbStack: wss://condescending_solomon.orb.local/ (WebSocket Secure)

Both WebSocket connections established successfully and functioned normally. The echo server handled both WS and WSS connections gracefully. This proved that OrbStack’s reverse proxy handles WebSockets correctly - the issue was specific to Jekyll, not OrbStack.

The Real Issue: Jekyll’s WebSocket Limitation

With OrbStack ruled out as the culprit, I investigated Jekyll-specific WebSocket issues and found this isn’t an isolated problem. Jekyll GitHub issue #9495 documents the exact same error occurring when Jekyll runs behind reverse proxies like Caddy.

The issue report contains a crucial finding from direct testing of Jekyll’s WebSocket behavior:

• Plain WebSocket (ws://) connections work fine
• Secure WebSocket (wss://) connections crash Jekyll

The report explains: “The Jekyll LiveReload barfs when it receives the encrypted data, because it can’t parse it.” Most importantly, the investigation proved this was caused by a specific Jekyll change: “I can confirm that [reverting PR #8718] fixes this issue; LiveReload works as expected even over reverse-proxied HTTPS URLs.” This definitively links the problem to Jekyll 4.3.2’s protocol detection change.

Understanding Jekyll’s Change in PR #8718

The issue stems from a change in Jekyll 4.3.2 (PR #8718) that modified how LiveReload script injection works.

Before Jekyll 4.3.2:

• Jekyll always used hardcoded http:// for LiveReload connections
• Even HTTPS pages made plain WebSocket (ws://) connections to LiveReload
• This worked fine with reverse proxies

After Jekyll 4.3.2:

• Jekyll switched to using location.protocol to dynamically detect the page protocol
• HTTPS pages now attempt WebSocket Secure (wss://) connections to LiveReload
• This breaks Jekyll’s WebSocket server

Here is the diff of the change in the PR #8718:

diff --git a/lib/jekyll/commands/serve/servlet.rb b/lib/jekyll/commands/serve/servlet.rb
index 2544616adf3..cee9c663a8a 100644
--- a/lib/jekyll/commands/serve/servlet.rb
+++ b/lib/jekyll/commands/serve/servlet.rb
@@ -101,7 +101,7 @@ def template
           @template ||= ERB.new(<<~TEMPLATE)
             <script>
               document.write(
-                '<script src="http://' +
+                '<script src="' + location.protocol + '//' +
                 (location.host || 'localhost').split(':')[0] +
                 ':<%=@options["livereload_port"] %>/livereload.js?snipver=1<%= livereload_args %>"' +
                 '></' +

While this seemed like a logical improvement for HTTPS sites, it revealed that Jekyll’s WebSocket implementation has a fundamental limitation: it cannot handle WSS connections.

Other WebSocket implementations (like the echo server) handle both WS and WSS connections gracefully, but Jekyll’s LiveReload server is limited to plain WebSocket connections only.

The Architecture Problem

The incompatibility creates a fundamental mismatch in modern HTTPS development environments:

Browser (HTTPS page) → wants WSS connection
     ↓
Jekyll LiveReload → only supports WS connection
     ↓
= Protocol incompatibility → Crash

When accessing Jekyll through any HTTPS reverse proxy:

1. Browser sees HTTPS page → uses location.protocol to determine WebSocket protocol
2. Browser attempts WSS connection to match the secure page protocol
3. Jekyll’s EventMachine parser receives encrypted data but can only handle plain text
4. Parser crashes trying to interpret encrypted frames as plain WebSocket data

Impact on Development Environments

This limitation affects Jekyll’s compatibility with modern development setups:

• Any HTTPS reverse proxy (OrbStack, Caddy, nginx, Traefik)
• Container orchestration (Docker Compose with HTTPS, Kubernetes ingress)
• Development tools that provide automatic HTTPS (like OrbStack’s domains)
• Production-like environments where HTTPS is standard

Jekyll 4.3.2+ is fundamentally incompatible with HTTPS development environments that use LiveReload. The previous hardcoded http:// approach accidentally worked around this limitation by forcing plain WebSocket connections even from HTTPS pages.

Time to Move On from Jekyll?

Jekyll’s popularity has been steadily declining as modern static site generators like Astro, Next.js, and Nuxt offer better developer experiences and more robust tooling. This WebSocket limitation is just another nail in the coffin—when your development server can’t handle basic HTTPS environments that have been standard for years, it’s a pretty clear sign the project is falling behind.

The choice between useful development features (like OrbStack’s convenient .orb.local domains) and Jekyll compatibility shouldn’t even be a question. Modern tools should adapt to modern workflows, not force developers to work around decade-old limitations.

At some point, you have to ask: is it worth sticking with a static site generator that breaks when you try to use it with contemporary development practices? Jekyll had its moment, but that moment was 2014. Maybe it’s time to migrate my website to something that actually works with today’s containerized, HTTPS-first development environments.

In last post, I’ve shown how to expose Docker’s API (port 2376) to your Tailscale VPN network on Linux systems. This post is a continuation of that guide, but for Windows with WSL2 backend.

Overview

This guide shows how to expose Docker’s API (port 2375) to your Tailscale VPN network on Windows with WSL2 backend. It’s a simple guide for those who want to use Docker remotely or access the status of Docker Containers from another device.

WARNING

We’re using port 2375 without SSL/TLS encryption. Thus, you need to use a VPN tunnel to provide encryption and access control.

Prerequisites

• Docker Desktop for Windows with WSL2 backend
• Tailscale VPN installed and running
• Administrator access to Windows

Step 1: Enable Docker API in Docker Desktop

1. Open Docker Desktop
2. Go to Settings → General
3. Check “Expose daemon on tcp://localhost:2375 without TLS”
4. Click Apply & Restart

Step 2: Find Your Tailscale IP Address

ipconfig

Look for the Tailscale adapter - note the IPv4 address (e.g., 100.xxx.xxx.xxx)

Step 3: Set Up Port Forwarding (Run as Administrator)

Open PowerShell as Administrator:

• Press Win + X → Select “Windows PowerShell (Admin)”
• Or press Win + R → type powershell → press Ctrl + Shift + Enter

Run these commands:

:: Add port forwarding rule (replace with your Tailscale IP)
netsh interface portproxy add v4tov4 listenport=2375 listenaddress=100.xxx.xxx.xxx connectport=2375 connectaddress=127.0.0.1

:: Configure Windows Firewall (Optional but recommended)
netsh advfirewall firewall add rule name="Docker API Tailscale" dir=in action=allow protocol=TCP localport=2375

Step 4: Verify Setup

:: Check port forwarding rules
netsh interface portproxy show all

Expected output:

Listen on ipv4:             Connect to ipv4:
Address         Port        Address         Port
--------------- ----------  --------------- ----------
100.xxx.xxx.xxx 2375        127.0.0.1       2375

Step 5: Test Connection

From any device on your Tailscale network:

# Test Docker connection
docker -H tcp://100.xxx.xxx.xxx:2375 version

# Or test port connectivity
telnet 100.xxx.xxx.xxx 2375

Cleanup Commands (if needed)

:: Remove port forwarding rule
netsh interface portproxy delete v4tov4 listenport=2375 listenaddress=100.xxx.xxx.xxx

:: Remove firewall rule
netsh advfirewall firewall delete rule name="Docker API Tailscale"

Security Notes

• ⚠️ Warning: This exposes Docker daemon without TLS encryption
• ✅ Safe: Tailscale provides encrypted VPN tunnel
• 🔒 Access: Only devices on your Tailscale network can connect
• 💡 Tip: Tailscale IPs are usually stable and don’t change frequently

Usage Examples

Once configured, you can use Docker remotely:

# Set environment variable for easier use
export DOCKER_HOST=tcp://100.xxx.xxx.xxx:2375

# Now use Docker commands normally
docker ps
docker images
docker run hello-world

Troubleshooting

• Connection refused: Check if Docker Desktop is running
• Port not accessible: Verify firewall rules and port forwarding
• Permission denied: Ensure commands were run as Administrator
• Tailscale IP changed: Update port forwarding rule with new IP

This guide shows how to expose Docker’s API (port 2376) to your Tailscale VPN network on Linux systems. It’s a simple guide for those who want to use Docker remotely or access the status of Docker Containers from another device.

WARNING

We’re using port 2376 but without SSL/TLS encryption. Thus, you need to use a VPN tunnel to provide encryption and access control.

Prerequisites

• Docker installed and running on Linux
• Tailscale VPN installed and connected
• sudo/root access to the system

Step 1: Find Your Tailscale IP Address

# Get your Tailscale IP
tailscale ip -4

Note the IP address (e.g., 100.xxx.xxx.xxx) - you’ll need this later.

Step 2: Configure Docker Daemon

Using systemd Override (Recommended)

# Create systemd override
sudo systemctl edit docker.service

Add this configuration (replace 100.xxx.xxx.xxx with your actual Tailscale IP):

[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H unix:///var/run/docker.sock -H tcp://100.xxx.xxx.xxx:2376 --containerd=/run/containerd/containerd.sock

Step 3: Apply Configuration

# Reload systemd and restart Docker
sudo systemctl daemon-reload
sudo systemctl restart docker

# Verify Docker is running
sudo systemctl status docker

Step 4: Configure Firewall

For UFW (Ubuntu/Debian):

# Install UFW if not present
sudo apt install ufw -y

# Allow SSH (important - don't lock yourself out!)
sudo ufw allow ssh

# Allow existing services
sudo ufw allow 80
sudo ufw allow 443

# Allow Docker API from Tailscale network
sudo ufw allow from 100.64.0.0/10 to any port 2376

# Enable UFW
sudo ufw enable

# Check status
sudo ufw status numbered

For iptables (Direct method):

# Add rule to allow Docker API from Tailscale network
sudo iptables -I INPUT -p tcp --dport 2376 -s 100.64.0.0/10 -j ACCEPT

# Save rules (method varies by distribution)
# Ubuntu/Debian:
sudo iptables-save > /etc/iptables/rules.v4

# CentOS/RHEL:
sudo service iptables save

Step 5: Verify Setup

# Check if Docker is listening on the correct port
sudo ss -tlnp | grep :2376

# Check your Tailscale IP
tailscale ip -4

# Test local connection
curl http://$(tailscale ip -4):2376/containers/json

Expected output should show JSON with container information.

Step 6: Test from Remote Device

From another device on your Tailscale network:

# Test Docker connection (replace with your Tailscale IP)
curl http://100.xxx.xxx.xxx:2376/containers/json

# Or test with Docker client
docker -H tcp://100.xxx.xxx.xxx:2376 ps

Security Notes

• ⚠️ Warning: This exposes Docker daemon without TLS encryption
• ✅ Safe: Tailscale provides encrypted VPN tunnel
• 🔒 Access: Only devices on your Tailscale network can connect
• 💡 Firewall: Uses Tailscale’s CGNAT range (100.64.0.0/10) for access control
• 🛡️ Best Practice: Consider using TLS certificates for production environments

Usage Examples

Once configured, you can use Docker remotely:

# Set environment variable for easier use (replace with your Tailscale IP)
export DOCKER_HOST=tcp://100.xxx.xxx.xxx:2376

# Now use Docker commands normally
docker ps
docker images
docker run hello-world

# Use with docker-compose remotely
docker-compose ps
docker-compose logs

Troubleshooting

Docker not listening on port 2376:

# Check Docker process
ps aux | grep dockerd

# Check systemd service
sudo systemctl cat docker.service | grep ExecStart

# View Docker logs
sudo journalctl -u docker.service -f

Connection refused from remote:

# Check firewall rules
sudo ufw status
# or
sudo iptables -L -n | grep 2376

# Test local connection first
curl http://localhost:2376/containers/json

Tailscale connectivity issues:

# Check Tailscale status
tailscale status

# Restart Tailscale
sudo systemctl restart tailscaled

# Re-authenticate if needed
sudo tailscale up

Docker service won’t start:

# Check for configuration conflicts
sudo journalctl -u docker.service --no-pager -l

# Temporarily remove custom config
sudo mv /etc/docker/daemon.json /etc/docker/daemon.json.backup
sudo systemctl restart docker

A long time ago, on a remote server far, far away…

When working with remote servers from macOS, you’ll inevitably encounter the dark side of the Force: hidden system files that follow your every move. These invisible menaces can break deployment scripts, bloat repositories, and generally cause chaos in your otherwise pristine remote environments.

EPISODE V: THE .DS_STORE STRIKES BACK

The Dark Side of macOS

Imperial .DS_Store files have driven Rebel developers from their remote server folders. These hidden files spread like the Dark Side across every folder you visit.

EPISODE VI: THE .DS_STORE STRIKES BACK

• Darth .DS_Store: “Your lack of defaults write is disturbing. My hidden files will spread across every folder you visit.”

• Luke: “But I’ve tried .gitignore! It has no power here on remote connections!”

• Yoda: “Use the Terminal, Luke. Or the sacred command to disable .DS_Store on network volumes.”

• Han: “I’ve been running from these hidden files for ten years. Not a remote server is safe in the galaxy!”

The Solution: Clean Up the Dark Side

1. Remove Existing .DS_Store Files

   # Clean up all .DS_Store files
   find . -name ".DS_Store" -delete
   

2. Prevent Future .DS_Store Creation

   # Disable .DS_Store on network volumes
   defaults write com.apple.desktopservices DSDontWriteNetworkStores -bool TRUE
   
   # Restart Finder to apply changes
   killall Finder
   

EPISODE VI: THE RETURN OF THE METADATA (._*)

“The ._* files are back, and they’re more annoying than ever.”

!!! warning “The Hidden Menace” When you copy files to a remote server, macOS creates mysterious ._* files. They’re like the Ewoks of the file system - small, seemingly harmless, but they can cause big problems.

• Darth Metadata: “Your files are not complete without my metadata. I will follow them everywhere.”

• Luke: “But these ._* files are causing issues with my Python scripts!”

• Yoda: “Hidden they are, but dangerous they can be. Clean them you must.”

The Solution: Defeat the Metadata Menace

1. Remove Existing Metadata Files

   # Remove all ._ files
   find . -name "._*" -delete
   
   # Or clean both .DS_Store and ._ files
   find . -type f -name "._*" -o -name ".DS_Store" -delete
   

2. Understanding the ._* Files - The Unstoppable Force

   On macOS, preventing the creation of ._* files (AppleDouble metadata) entirely is not officially supported—especially on non-HFS+ or non-APFS volumes.

   Why ._* files are created: macOS uses ._* AppleDouble files to store:

   - Resource forks
   - Extended attributes (e.g., custom icons, tags)
   - Finder metadata
   
   These are automatically created when copying files to filesystems that don't support extended attributes, such as:
   - SMB shares (Linux Samba servers)
   - FAT, exFAT, NTFS drives
   - Some WebDAV volumes
   

3. The Harsh Reality

   Just kidding, there are no best available solutions for this problem. That’s why using Finder to mount remote servers as local drives is not an elegant solution (Check this.

4. For Git Users

   Add the following to your .gitignore file:

   # Add to your .gitignore (won't prevent creation, but prevents tracking)
   echo "._*\n.DS_Store" >> .gitignore
   

Final Words

“The Force is strong with clean file systems.”

May your remote development be free of metadata files, and may the Force be with you! 🚀

VS Code Remote SSH is banned

QUT Aqua banned VS Code Remote SSH extension due to potential high workload on the node. Even you try to connect to Aqua through Remote SSH, it will be disconnected automatically after around 30 seconds. Check this for more details.

So… you’re trying to develop on QUT Aqua, but the server gods have other plans. Maybe you can’t use VS Code Remote SSH. Maybe you’re just feeling adventurous. But do not worry — you can still edit remote files and develop like a champ. Here’s how I’ve kept my sanity while developing on remote HPC systems.

Before you start

Recommend to add a shortcut to ~/.ssh/config

If you are using SSH keys to connect to the HPC, you can add a shortcut to ~/.ssh/config to make your life easier. QUT Aqua documentation provides a guide on how to set up SSH keys for passwordless login.

# Add to your ~/.ssh/config
Host aqua
    HostName aqua.qut.edu.au
    User your-username
    IdentityFile ~/.ssh/id_rsa_aqua # Add your SSH key here
    ServerAliveInterval 60

Then, you can connect to the HPC by running ssh aqua. Also, you can use aqua to replace your-username@aqua.qut.edu.au in the following commands.

🧩 1. Fake it with SSH-mounted folders

🧀 Option A: Mount via Finder

Here’s a quick guide for macOS users. Please refer to the QUT Aqua documentation for other OS.

1. Open Finder → Go → Connect to Server...

2. Enter:

   smb://hpc-fs/home/
   

3. Mount it, then open the folder in VS Code like it’s 1999.

📝 Note: You can edit files, but no shell, no Git, and no terminal tantrums. It’s like eating cake without the frosting.

But … is this method elegant?

You’ve mounted an SMB share to your Finder. While this works, there are some significant limitations:

1. Git Limitations - Your version control system becomes severely limited over SMB. Git operations that work fine locally will fail or behave unpredictably through the mounted share.
2. VS Code Terminal Issues - The integrated terminal in VS Code won’t work properly with mounted SMB shares. You’ll get Command not found errors for most terminal operations.
3. Connection Stability - SMB connections can drop unexpectedly, especially during longer work sessions or when the network is unstable.
4. HPC Dependency - Since all your files live on the server, any HPC maintenance or downtime makes your work completely inaccessible.
5. The .DS_Store Problem (macOS) - Your Mac will create .DS_Store files in every folder you visit through Finder. These desktop service files clutter the HPC filesystem and serve no purpose on the server.

🔧 Option B: SSHFS — Mount through SSH Wizardry

Mount your HPC home directory directly via SSH, no Finder fluff. It’s like having your HPC filesystem in your pocket.

For macOS Users:

# Install the prerequisites (because your Mac doesn't come with everything, despite what Apple claims)
brew install macfuse
brew install gromgit/fuse/sshfs-mac

# Mount your HPC home (1)
mkdir ~/aqua
sshfs your-username@aqua.qut.edu.au:/home/your-username ~/aqua #(2)

# When you're done pretending these files are local
umount ~/aqua
# Or if that fails spectacularly (as technology loves to do)
diskutil unmount ~/aqua

Notes:

1. When you’re running sshfs first time, you will be asked to go to “System Preferences” → “Security & Privacy” → “Security” → click “Allow” for running the app. Then you also need to restart your Mac.
2. You can use aqua to replace your-username@aqua.qut.edu.au if you have added a shortcut to ~/.ssh/config.

For Linux Users (Ubuntu):

# Install SSHFS (because of course Linux makes you work for everything)
sudo apt install sshfs

# Mount your HPC home, telling the laws of physics to take a break
mkdir -p ~/aqua
sshfs your-username@aqua.qut.edu.au:/home/your-username ~/aqua -o follow_symlinks

# To send these files back to their natural habitat
fusermount -u ~/aqua

For Windows Users:

Install WinFSP and SSHFS-Win, because Windows needs two separate things to do what other systems accomplish with one. Then use Windows Explorer (which Microsoft keeps renaming as if that will make us forget its bugs) to map a network drive:

\\sshfs\your-username@aqua.qut.edu.au

Then open it in VS Code like you’ve just performed a miracle:

code ~/aqua

✅ Pro: Looks local. Feels local. Git operations work… until they mysteriously don’t

❌ Con: Feels too local for large files. Might lag. If the connection drops, your filesystem freezes like it’s seen a ghost

Performance Tips That Might Help (No Promises)

• Use -o cache=yes to create the illusion of performance (side effects may include file synchronization existential crises)
• Add -o compression=yes to squeeze your data through the internet tubes more efficiently
• If everything hangs, adjust your ServerAlive settings, which is like giving your connection a gentle nudge every few minutes to check if it’s still breathing

Working with Git Over SSHFS: A Tragicomedy

When using Git over SSHFS, you’re essentially asking Git to perform a synchronized swimming routine while blindfolded. For anything more complex than a simple commit, consider SSH-ing directly into the server and running Git commands there. Your future self will thank you for not testing the limits of your patience.

🔄 2. rsync, scp and git: Your old-school sync buddies

⚙️ Option A: rsync & scp — The Reliable Workhorse

# Sync your local code to HPC
rsync -avz ./my-project/ your-username@aqua.qut.edu.au:/home/your-username/projects/

# Sync back from HPC
rsync -avz your-username@aqua.qut.edu.au:/home/your-username/projects/ ./my-project/

Or for a quick one-file fling:

scp script.py your-username@aqua.qut.edu.au:/home/your-username/projects/

It’s not fancy, but it works — like duct tape.

⚡ Option B: Git — The Version Control Way

If you are version-controlling your life (as you should), Git is a clean and reliable method.

# On your local machine
git init
git add .
git commit -m "Initial commit"
git remote add aqua your-username@aqua.qut.edu.au:/path/to/repo
git push aqua main

# On the HPC
git clone your-username@aqua.qut.edu.au:/path/to/repo

✅ Pro: Clean history, branch control, reproducibility

❌ Con: Needs initial setup and your SSH keys must behave

🖥️ 3. The Terminal-Only Approach

When all else fails, embrace the terminal:

ssh your-username@aqua.qut.edu.au

Then pick your weapon of choice:

• vim — For the brave
• nano — For the sane
• neovim — For the modern
• emacs — For the… unique

🎯 Bonus: Fast, keyboard-driven, and doesn’t require GUI permission forms.

Note: I will write another page about how to use neovim and its plugins to replace VS Code as a lightweight editor (with SSH).

🌐 4. The Web-Based Approach

📓 Option A: Jupyter Notebooks

Install Jupyter Lab in HPC before you start

QUT Aqua documentation provides a guide on how to install Miniconda in HPC.

# On the HPC
# I prefer to use Jupyter Lab instead of Jupyter Notebook
jupyter lab --no-browser --port=8888 # (1)

# On your local machine, forward the port 8888 to your local machine
# local_port:localhost:remote_port (2)
ssh -N -L 8888:localhost:8888 your-username@aqua.qut.edu.au

Notes:

1. If port 8888 is already in use, you can try another port, e.g. 8889.
2. -N means no command to run on the remote machine. -L means forward the local port to the remote port. Both local and remote ports are 8888 in this case.

🔥 Option B: VSCode in Browser

This might require a sysadmin’s blessing!

Fortunately, the server gods haven’t locked everything down:

1. Install code-server on the HPC.

   # On HPC server
   # Install code-server to your home directory
   curl -fsSL https://code-server.dev/install.sh | sh -s -- --method standalone --prefix=$HOME
   # code-server will be installed to $HOME/bin/code-server
   
   # check if code-server is installed
   code-server --version
   
   # Start code-server
   code-server  --bind-addr 127.0.0.1:8080 --disable-telemetry --disable-update-check --auth none
   
   # On your local machine
   # Forward the port 8080 to your local machine
   ssh -N -L 8080:127.0.0.1:8080 your-username@aqua.qut.edu.au
   

2. Open it in your browser

   # Open the web page in your browser
   http://localhost:8080
   

3. Marvel as VS Code rises from the ashes — web-style

Sync VS Code settings to code-server

You can import your VS Code settings to code-server by importing the profile from VS Code. Check out this page for more details about how to export and import profiles. However, this’s not the perfect solution. Not all VS Code extensions are available for code-server, some extensions are restricted for Microsoft VS Code. Only the extensions that are available for code-server are listed in Open VSX Registry.

Run code-server in the background with tmux

You can run code-server in the background with tmux to avoid the session being killed after you disconnect from the HPC.

# Start a new tmux session
tmux new -s code

# Run code-server in the background
code-server --bind-addr 127.0.0.1:8080 --disable-telemetry --disable-update-check --auth none

# Detach from the tmux session: `Ctrl+b`, then `d`

# Reattach to the tmux session
tmux attach -t code

# Kill the tmux session
tmux kill-session -t code

# If you forget the session name, you can list all sessions
tmux ls

Known issue on Integrated Terminal and Extension Host

I found that the terminal and the extension host are not stable when using code-server. The issue seems to revolve around the ptyHost, File Watcher, and Extension Host, and it’s being repeatedly killed by SIGTERM.

🧠 What Is Happening?

[12:18:01] ptyHost terminated unexpectedly with code null
[12:18:01] [File Watcher (universal)] restarting watcher after unexpected error: terminated by itself with code null, signal: SIGTERM (ETERM)
[12:18:01] [127.0.0.1][d0f383fd][ExtensionHostConnection] <3126357> Extension Host Process exited with code: null, signal: SIGTERM.
[12:18:02] [127.0.0.1][d0f383fd][ExtensionHostConnection] Unknown reconnection token (seen before).
[12:18:02] [127.0.0.1][368c67ad][ExtensionHostConnection] New connection established.
[12:18:02] [127.0.0.1][368c67ad][ExtensionHostConnection] <3132486> Launched Extension Host Process.

📜 From the logs:

• 💥 The ptyHost process (responsible for terminal sessions) crashed or was killed — possibly due to system resource limits or policy.
• 📦 File watcher was forcefully killed (SIGTERM) — system or job policy likely did this.
• 🧩 Extension host was also killed — same reason, likely tied to HPC rules.
• ↩️ code-server tried to reconnect to the crashed extension host but failed.
• 🆕 code-server restarted the extension host process automatically.

TL;DR — What Works (and What Requires Sacrifice)

Final Words

Remote development on HPC doesn’t have to be a pain. Pick your poison, set up your workflow, and remember: the best development environment is the one that doesn’t make you want to throw your computer out the window.

Happy coding, and may your HPC connections be stable! 🚀