Guidebolt

Website Development

2018.04.20

The fundamentals.

TLS Certificate

Database

Website Architecture

Practical Development Path Overview

Web Dev Checklist

It is better to run and scale a small solid system than to maintain and drag a large sprawling mess.

Development, Scaling, Monitoring, Hardening

The 4 key work categories are application development (getting things working, adding features, improving UI/UX), application scaling (load balanced app servers, DNS round robin, master-slave databases), application monitoring (detailed logging, server performance history), and application hardening (code refactoring, authentication security, database replication/backups, disaster recovery preparation).

Synchronous Dependencies

You have to be careful with synchronous response dependencies, especially from external web API providers. In case their APIs are down, you want to keep the connection timeout short in order to avoid web server memory overload on your side. In some cases, critical API requests must have a long timeout; in these cases, you want to look at asynchronous solutions such as inserting a task into a message queue (for some worker to perform asynchronously) and just tell the user in the moment that their request has been successfully received for processing.

Tasks are split across front-end (layout, style), back-end (application script, database), and systems administration (server setup, networking).

Generate a SSH access keypair with this script: RSA 4096-bit Keypair Generation | Bash Script

Pick a server hosting provider like Digital Ocean or Amazon Web Services.

Spin up a server with your public key, noting the server's own public key fingerprint.

Recommended Operating System: Ubuntu or Centos (use the long-term support versions)

Configure and enable the firewall.

Standard Outbound Settings: Allow All

Standard Inbound Settings: Allow Ports 80 (HTTP), 443 (HTTPS), 22 (SSH)

Install the fundamental packages.

Recommended Web Server: Nginx

Recommended Version Control: Git

Recommended Scripting Language: PHP or Python

Use an application framework like Laraval (PHP front/back full framework, good for general web applications) or Flask (Python back-end microservice framework, good for API services or scaling an existing app with separation of responsibilities).

Recommended In-Memory Database: Redis

Recommended Load Balancer: HAProxy or Nginx

Use containers to scale with rapid, uniform server deployment, which is useful for creating auto-scaling clusters and replacement systems. Containers are distinct from normal server deployment scripts in that containers use server images that are an exact copy of an original server state, whereas scripts generally grab new data from external sources and re-run installation processes.

Use central environment servers to set flexible environment variables across multiple server clusters. To do so, application servers are deployed with an environment reload command to grab environment variables from the environment server cluster by calling the environment cluster's load balancer private IP or private domain name. On application server deployment and after every change of an environment variable, make the application cluster supervisor issue an order to every application server to activate their environment reload command.

Clean, auto-scaling systems can be achieved by combining application microservices made of containers that draw from central environment servers, central log servers, central database clusters, auto-updating load balancers, and an organized networking infrastructure. Be warned; this is a late-game design that must be approached at an appropriate pace based on the scaling trajectory of need and cost.

Zero Down-time Changes

Zero down-time application and database changes must be designed and sectioned into distinct, well-ordered, backwards-compatible updates that allow for the gradual re-coloration of a server cluster.

Application Example: If an application update changes the encoding format of a stored phone number, then the database will soon contain phone numbers encoded with both the old and new format. Thus that same update must also change the decoding process to support both the old and new formats. After the first application update, the database must be manually scripted to transform all remaining old-format phone numbers into the new format. Then a second application update may be rolled out to refine the decoding process to only support the new encoding format.

Database Example: Suppose a database table must be replaced by a new table. If a new table is created then populated by data from the old table, then the new table will miss all the new data generated between the time when the old table data is copied and the time when the application is switched to read from the new table. This simple approach might work for small regional applications when done at zero traffic periods, but high-frequency international applications must use a more refined strategy (the correct way). First, create the new table then make an application update that causes new data to be written to both tables (old and new). Second, manually populate the new table with data from the old table. Third, make an application update to read data from the new table and stop writing to the old table. Fourth, remove and archive the old table. This general procedure ensures continuous, error-free operation across every step of the database change.

Control Ladder

Network Control - The infrastructure through which users send and receive data packets. Nameservers, regional ISP lines, network firewalls,

Server Control - The hardware on which your direct or virtual server runs. Shell access, server firewall, system users, file and folder permissions.

Code Control

Data Control

Website Analysis

Speed: Google PageSpeed Analyzer

Security: Qualys SSL Labs Server Test

SEO: SiteCheckup Test

Blacklist: MXToolbox Blacklist Check

Fast Website Reference | Comedy

Architecture

Application-Database Isolation

Application Server Farm/Standbys

Application Load Balancer

Database Replication

Database Backups

Database Load Balancer

Multi-region Servers

Geographic Localization

DNS Scaling

In-Memory Database

Message Queue

System Status Monitor

See examples below:

Stripe System Status Page

Google System Status Page

Concepts

API Type: SOAP vs JSON