README.md from short-d/short

README.md
Summary

Maintainability

Test Coverage

Issues
# Short
 
[![Build Status](https://ci.time4hacks.com/api/badges/short-d/short/status.svg)](https://ci.time4hacks.com/short-d/short)
[![codecov](https://codecov.io/gh/short-d/short/branch/master/graph/badge.svg)](https://codecov.io/gh/short-d/short)
[![Maintainability](https://api.codeclimate.com/v1/badges/910f974653f1b3495534/maintainability)](https://codeclimate.com/github/short-d/short/maintainability)
[![Go Report Card](https://goreportcard.com/badge/github.com/short-d/short)](https://goreportcard.com/report/github.com/short-d/short)
[![MIT License](https://img.shields.io/badge/License-MIT-blue.svg)](https://github.com/byliuyang/short)
[![Floobits Status](https://floobits.com/byliuyang/short.svg)](https://floobits.com/byliuyang/short/redirect)
 
![Demo](promo/marquee.png)
 
## Preview
 
![Demo](doc/demo.gif)
 
## Get `s/` Chrome extension
 
Install it from [Chrome Web Store](https://short-d.com/r/ext) or build it
from [source](https://short-d.com/r/ext-code)
 
## Dependent Projects
 
- [app](https://github.com/short-d/app): Reusable framework for Go apps & command
   line tools.
- [kgs](https://github.com/short-d/kgs): Offline unique key generation service.
 
## Table of Contents
 
1. [Getting Started](#getting-started)
   1. [Accessing the source code](#accessing-the-source-code)
   1. [Prerequisites](#prerequisites)
   1. [Local environmental variables](#local-environmental-variables)
   1. [Create reCAPTCHA account](#create-recaptcha-account)
   1. [Configure Single Sign On](#configure-single-sign-on)
   1. [Backend](#backend)
   1. [Frontend](#frontend)
1. [System Design](#system-design)
   1. [App Level Architecture](#app-level-architecture)
   1. [Service Level Architecture](#service-level-architecture)
   1. [Object Oriented Design](#object-oriented-design)
   1. [Dependency Injection](#dependency-injection)
   1. [Database Modeling](#database-modeling)
   1. [Feature Toggle](#feature-toggle)
   1. [Search Engine Optimization](#search-engine-optimization)
   1. [Social Media Summary Card](#social-media-summary-card)
1. [Testing](#testing)
   1. [The Importance Of Automation](#the-importance-of-automation)
   1. [Testing Strategy](#testing-strategy)
   1. [Unit Testing](#unit-testing)
   1. [Integration Testing](#integration-testing)
   1. [Component Testing](#component-testing)
   1. [Contract Testing](#contract-testing)
   1. [End To End Testing](#end-to-end-testing)
   1. [The Test Pyramid](#the-test-pyramid)
1. [Deployment](#deployment)
   1. [Continuous Delivery](#continuous-delivery)
   1. [Kubernetes](#kubernetes)
   1. [GitOps](#gitops)
1. [Tools We Use](#tools-we-use)
1. [Contributing](#contributing)
1. [Author](#author)
1. [License](#license)
 
## Getting Started
 
### Accessing the source code
 
```bash
git clone https://github.com/short-d/short.git
```
 
### Prerequisites
 
- [Go](https://golang.org/doc/install) v1.13.1
- [Node.js](https://nodejs.org/en/download/) v12.12.0
- [Yarn](https://classic.yarnpkg.com/en/docs/install) v1.19.1
- [PostgreSQL](doc/tutorial/POSTGRES.md) v12.0
 
### Local environmental variables
 
1. Copy `backend/.env.dist` file to `backend/.env`:
 
   ```bash
   cp backend/.env.dist backend/.env
   ```
 
1. Copy `frontend/.env.development.dist` file to `frontend/.env.development`:
 
   ```bash
   cp frontend/.env.development.dist frontend/.env.development
   ```
 
Multiple consecutive blank lines
 
### Create reCAPTCHA account
 
1. Sign up at [ReCAPTCHA](https://short-d.com/r/recaptcha) with the
   following configurations:
 
   | Field           | Value          |
   |-----------------|----------------|
   | Label           | `Short`        |
   | reCAPTCHA type  | `reCAPTCHAv3`  |
   | Domains         | `localhost`    |
 
1. Open `settings`. Copy `SITE KEY` and `SECRET KEY`.
 
1. Replace the value of `RECAPTCHA_SECRET` in the `backend/.env` file with
   `SECRET KEY`.
1. Replace the value of `REACT_APP_RECAPTCHA_SITE_KEY` in
   `frontend/.env.development` file with `SITE KEY`.
 
Headers should be surrounded by blank lines
### Configure Single Sign On
#### Google
 
Create a new Client ID at
   [Google API Credentials](https://console.developers.google.com/apis/credentials):
 
1. Click on `Create Credentials` and select `OAuth client ID`.
 
1. Select `Web application` for `Application type`.
 
Line length
1. Fill in `http://localhost/oauth/google/sign-in/callback` for `Authorized redirect URIs` and click on `Create`.
Trailing spaces
 
Line length
1. Replace the value of `GOOGLE_CLIENT_ID` in `backend/.env` file with `Your Client ID`.
1. Replace the value of `GOOGLE_CLIENT_SECRET` in `backend/.env` file with
   `Your Client Secret`.
 
Headers should be surrounded by blank lines
#### Facebook
Trailing spaces
Line length
You can find the detailed instructions on setting up Facebook sign in [here](doc/sso/FACEBOOK.md) in case you are interested in. 
 
Headers should be surrounded by blank lines
#### Github
Line length
You can find the detailed instructions on setting up Github sign in [here](doc/sso/GITHUB.md) in case you are interested in.
Trailing spaces
   
Headers should be surrounded by blank lines
### Backend
 
1. Update placeholder values with your own configurations.
 
1. Launch backend server
 
   ```bash
   cd backend
   ./scripts/dev
   ```
 
1. Remember to install developers tools before start coding:
 
   ```bash
   ./scripts/tools
   ```
 
### Frontend
 
1. Update `REACT_APP_RECAPTCHA_SITE_KEY` in `frontend/.env.development`.
 
1. Launch frontend server
 
   ```bash
   cd frontend
   ./scripts/dev
   ```
 
1. Visit [http://localhost:3000](http://localhost:3000)
 
## System Design
 
### App Level Architecture
 
Short backend is built on top of
[Uncle Bob's Clean Architecture](https://api.short-d.com/r/ca), the central
objective of which is separation of concerns.
 
![Short Backend](doc/clean-architecture/short-backend.jpg)
 
It enables the developers to modify a single component of the system at a time
while leaving the rest unchanged. This minimizes the amount of changes have to
be made in order to support new requirements as the system grows. Clean
Architecture also improves the testability of system, which in turn saves
precious time when creating automated tests.
 
### Service Level Architecture
 
Short adopts [Microservices Architecture](https://api.short-d.com/r/ms) to
organize dependent services around business capabilities and to enable
independent deployment of each service.
 
![Short Cloud](doc/cloud/overall.jpg)
Line length
Trailing spaces
[SSR](https://docs.google.com/document/d/16iV91aESfnYU6rIEWGEzws3nbDX3hB-St9gAxrtCAa8), 
[Toggle](https://docs.google.com/document/d/1TuWexeKwhQh8JTytRAwST3XujBi0wTGExwJan-WfXWs),
Line length
[Status Page](https://docs.google.com/document/d/1pgRNnD8yAlEmj-sucS_FZ89LdvBy5zpKQ9OvILoBqDM), Search,
[Data Reporter](https://docs.google.com/document/d/1-BtxBuS4zIk8H1oXDe-qqEccWp4v6aT2GrWBfwIX5oI),
[Feedback Widget](https://docs.google.com/document/d/1IoaTMHsOi5Tb0ZV4btxsvUnKplKi2lxaIYU600cwRuc),
 and Cloud API are still under active development.
 
### Object Oriented Design
 
Short leverages class design, package cohesion, and package coupling principles
to manage logical dependency between internal components.
 
#### Class Design
 
Line length
| Principal                                                        | Description                                                            |
|------------------------------------------------------------------|------------------------------------------------------------------------|
Line length
| [Single Responsibility Principle](https://api.short-d.com/r/srp) | A class should have one, and only one, reason to change.               |
Line length
| [Open Closed Principle](https://api.short-d.com/r/ocp)           | You should be able to extend a classes behavior, without modifying it. |
Line length
| [Liskov Substitution Principle](https://api.short-d.com/r/lsp)   | Derived classes must be substitutable for their base classes.          |
Line length
| [Interface Segregation Principle](https://api.short-d.com/r/isp) | Make fine grained interfaces that are client specific.                 |
Line length
| [Dependency Inversion Principle](https://api.short-d.com/r/dip)  | Depend on abstractions, not on concretions.                            |
#### Package Cohesion
 
Line length
| Principal                                                            | Description                                           |
|----------------------------------------------------------------------|-------------------------------------------------------|
Line length
| [Release Reuse Equivalency Principle](https://api.short-d.com/r/rep) | The granule of reuse is the granule of release.       |
Line length
| [The Common Closure Principle](https://api.short-d.com/r/ccp)        | Classes that change together are packaged together.   |
Line length
| [The Common Reuse Principle](https://api.short-d.com/r/crp)          | Classes that are used together are packaged together. |
#### Package Coupling
 
Line length
| Principal                                                       | Description                                           |
|-----------------------------------------------------------------|-------------------------------------------------------|
Line length
| [Acyclic Dependencies Principle](https://api.short-d.com/r/adp) | The dependency graph of packages must have no cycles. |
Line length
| [Stable Dependencies Principle](https://api.short-d.com/r/sdp)  | Depend in the direction of stability.                 |
Line length
| [Stable Abstractions Principle](https://api.short-d.com/r/sap)  | Abstractness increases with stability.                |
### Dependency Injection
 
Short produces flexible and loosely coupled code, by explicitly providing
components with all of the dependencies they need.
 
```go
type Authenticator struct {
  tokenizer          fw.CryptoTokenizer
  timer              fw.Timer
  tokenValidDuration time.Duration
}
 
func NewAuthenticator(
  tokenizer fw.CryptoTokenizer,
  timer fw.Timer,
  tokenValidDuration time.Duration,
) Authenticator {
  return Authenticator{
    tokenizer:          tokenizer,
    timer:              timer,
    tokenValidDuration: tokenValidDuration,
  }
}
```
 
Short also simplifies the management of the big block of order-dependent
initialization code with [Wire](https://api.short-d.com/r/wire), a compile time
dependency injection framework by Google.
 
```go
func InjectGraphQlService(
  name string,
  sqlDB *sql.DB,
  graphqlPath provider.GraphQlPath,
  secret provider.ReCaptchaSecret,
  jwtSecret provider.JwtSecret,
  bufferSize provider.KeyGenBufferSize,
  kgsRPCConfig provider.KgsRPCConfig,
  tokenValidDuration provider.TokenValidDuration,
) (mdservice.Service, error) {
  wire.Build(
    wire.Bind(new(fw.GraphQlAPI), new(graphql.Short)),
    wire.Bind(new(url.Retriever), new(url.RetrieverPersist)),
    wire.Bind(new(url.Creator), new(url.CreatorPersist)),
    wire.Bind(new(repo.UserURLRelation), new(db.UserURLRelationSQL)),
    wire.Bind(new(repo.URL), new(*db.URLSql)),
    wire.Bind(new(keygen.KeyGenerator), new(keygen.Remote)),
    wire.Bind(new(service.KeyFetcher), new(kgs.RPC)),
 
    observabilitySet,
    authSet,
 
    mdservice.New,
    provider.NewGraphGophers,
    mdhttp.NewClient,
    mdrequest.NewHTTP,
    mdtimer.NewTimer,
 
    db.NewURLSql,
    db.NewUserURLRelationSQL,
    provider.NewRemote,
    url.NewRetrieverPersist,
    url.NewCreatorPersist,
    provider.NewKgsRPC,
    provider.NewReCaptchaService,
    requester.NewVerifier,
    graphql.NewShort,
  )
  return mdservice.Service{}, nil
}
```
 
### Database Modeling
 
![Entity Relation Diagram](doc/db/er.jpg)
 
### Feature Toggle
 
Short employs `feature toggles` to modify system behavior without changing code.
UI components controlled by the feature toggles are created inside a centralized
`UIFactory` in order to avoid having nested `if` `else` statement across the
code base:
 
```typescript
// UIFactory.tsx
export class UIFactory {
  constructor(
    private featureDecisionService: IFeatureDecisionService
  ) {}
 
  public createGoogleSignInButton(): ReactElement {
    if (!this.featureDecisionService.includeGoogleSignButton()) {
      return <div />;
    }
    return (
      <GoogleSignInButton
        googleSignInLink={this.authService.googleSignInLink()}
      />
    );
  }
 
  public createGithubSignInButton(): ReactElement {
    if (!this.featureDecisionService.includeGithubSignButton()) {
      return <div />;
    }
    return (
      <GithubSignInButton
        githubSignInLink={this.authService.githubSignInLink()}
      />
    );
  }
}
```
 
Short also provides `IFeatureDecisionService` interface, allowing the developers
to switch to dynamic feature toggle backend in the future by simply swapping
the dependency injected.
 
```typescript
// FeatureDecision.service.ts
export interface IFeatureDecisionService {
  includeGithubSignButton(): boolean;
  includeGoogleSignButton(): boolean;
  includeFacebookSignButton(): boolean;
}
```
 
```typescript
// StaticConfigDecision.service.ts
import { IFeatureDecisionService } from './FeatureDecision.service';
 
export class StaticConfigDecisionService implements IFeatureDecisionService {
  includeGithubSignButton(): boolean {
    return false;
  }
  includeGoogleSignButton(): boolean {
    return false;
  }
  includeFacebookSignButton(): boolean {
    return true;
  }
}
```
 
```typescript
// dep.ts
export function initUIFactory(
  ...
): UIFactory {
  ...
  const staticConfigDecision = new StaticConfigDecisionService();
  ...
  return new UIFactory(
    ...,
    staticConfigDecision
  );
}
```
 
You can read about the detailed feature toggle design on
[this article](https://martinfowler.com/articles/feature-toggles.html).
 
### Search Engine Optimization
 
In order to improve the quality and quantity of the website's traffic, Short
increases its visibility to web search engines through HTML meta tags.
 
```html
<!-- ./frontend/public/index.html -->
<title>Short: Free online link shortening service</title>
 
<!-- Search Engine Optimization -->
<meta name="description"
      content="Short enables people to type less for their favorite web sites">
<meta name="robots" content="index, follow">
<link href="https://short-d.com" rel="canonical">
```
 
If you search `short-d.com` on Google, you should see Short shows up as
the first result:
 
![Google Search Result](doc/seo/google.jpg)
 
### Social Media Summary Card
 
#### Facebook & LinkedIn
 
Short leverages `Open Graph` tags to control what content shows up in
the summary card when the website is shared on Facebook or LinkedIn:
 
```html
<!-- ./frontend/public/index.html -->
<!-- Open Graph -->
<meta property="og:title" content="Short: Free link shortening service"/>
<meta property="og:description"
      content="Short enables people to type less for their favorite web sites"/>
<meta property="og:image"
      content="https://short-d.com/promo/small-tile.png"/>
<meta property="og:url" content="https://short-d.com"/>
<meta property="og:type" content="website"/>
```
 
Shared on Facebook:
 
![Facebook Card](doc/social-media-card/facebook.jpg)
 
Shared on LinkedIn:
 
![LinkedIn Card](doc/social-media-card/linkedin.jpg)
 
#### Twitter
 
Twitter uses its own meta tags to determine what will show up when
the website is mentioned in a Tweet:
 
```html
<!-- Twitter -->
<meta name="twitter:card" content="summary_large_image"/>
<meta name="twitter:site" content="@byliuyang11"/>
<meta name="twitter:title" content="Short: Free link shortening service"/>
<meta name="twitter:description"
      content="Short enables people to type less for their favorite web sites"/>
<meta name="twitter:image" content="https://short-d.com/promo/twitter-card.png"/>
```
 
![Twitter Card](doc/social-media-card/twitter.jpg)
 
## Testing
 
### The Importance Of Automation
 
Short is maintained by a small team of talented software engineers working
at Google, Uber, and Vmware as a side project. The team wants to deliver new
features faster without sacrificing its quality. Testing ever-increasing
amount of features manually soon becomes impossible — unless we want
to spend all our time with manual, repetitive work instead of delivering
working features.
 
Test automation is the only way forward.
 
### Testing Strategy
 
![Test Strategy](doc/testing/test-strategy.png)
 
Please read [Testing Strategies in a Microservice Architecture](https://martinfowler.com/articles/microservice-testing)
for a detailed introduction on test strategies.
 
### Unit Testing
 
A unit test exercises the smallest piece of testable software in the
application to determine whether it behaves as expected.
 
![Unit Test](doc/testing/unit-test.png)
 
Run unit tests for backend:
 
```bash
cd backend
./scripts/unit-test
```
 
#### Sociable And Solitary
 
![Two Types of Unit Test](doc/testing/unit-test-two-types.png)
 
#### The FIRST Principal
 
- [F]ast: Unit tests should be fast otherwise they will slow down
   development & deployment.
- [I]ndependent: Never ever write tests which depend on other test cases.
- [R]epeatable: A repeatable test is one that produces the same results
   each time you run it.
- [S]elf-validating: There must be no manual interpretation of the results.
- [T]imely/[T]horoughly: Unit tests must be included for every pull request
   of a new feature and cover edge cases, errors, and bad inputs.
 
#### Test Structure
 
An automated test method should be composed of 3As: Arrange, Act, and Assert.
 
- [A]rrange: All the data needed for a test should be arranged as part
  of the test. The data used in a test should not depend on the environment
  in which the test is running.
- [A]ct: Invoke the actual method under test.
- [A]ssert: A test method should test for a single logical outcome.
 
### Integration Testing
 
An integration test verifies the communication paths and interactions
between components to detect interface defects.
 
![Integration Test](doc/testing/integration-test.png)
 
Run integration tests for backend:
 
```bash
cd backend
./scripts/integration-test
```
 
### Component Testing
 
A component test limits the scope of the exercised software to a portion
of the system under test, manipulating the system through internal code
interfaces and using test doubles to isolate the code under test from
other components.
 
#### In Process
 
![Component Test](doc/testing/component-test-in-process.png)
 
#### Out Of Process
 
![Component Test](doc/testing/component-test-out-of-process.png)
 
### Contract Testing
 
An integration contract test is a test at the boundary of an external
service verifying that it meets the contract expected by a consuming
service.
 
### End To End Testing
 
An end-to-end test verifies that a system meets external requirements
and achieves its goals, testing the entire system, from end to end.
 
### The Test Pyramid
 
![Test Pyramid](doc/testing/test-pyramid.png)
 
## Deployment
 
Headers should be surrounded by blank lines
### Continuous Delivery
![Continuous Delivery](doc/deployment/cd-pipeline.jpg)
 
Trailing spaces
Currently, we use continuous delivery to deploy code changes to staging & 
Trailing spaces
production environment. 
 
Merging pull request into master branch on Github will automatically deploy the
changes to [staging](https://staging.short-d.com). Merging from `master` branch
to `production` branch will automatically deploy the latest code to the production.
 
Trailing spaces
In the future, when after we add enough automated tests, we may migrate to 
continuous deployment instead for faster releases.
 
Line length
You can find the differences between continuous delivery & continuous deployment [here](doc/tutorial/CI-CD.md)
 
### Kubernetes
 
Short leverages [Kubernetes](https://kubernetes.io) to automate deployment, scaling,
and management of containerized microservices.
 
![Node overview](https://d33wubrfki0l68.cloudfront.net/5cb72d407cbe2755e581b6de757e0d81760d5b86/a9df9/docs/tutorials/kubernetes-basics/public/images/module_03_nodes.svg)
 
Headers should be surrounded by blank lines
### GitOps
Short uses [GitOps](https://github.com/byliuyang/gitops) to configure Kubernetes
cluster and span up new services.
 
![Git Ops](doc/deployment/gitops.jpg)
 
## Tools We Use
 
- [Drone](https://short-d.com/r/ci): Continuous integration
  written in Go
- [Code Climate](https://codeclimate.com/): Automated code
  review
- [ElephantSQL](https://www.elephantsql.com): Managed PostgreSQL service.
 
## Contributing
 
Please read [CONTRIBUTING.md](CONTRIBUTING.md) for details on our code
of conduct, the process for submitting pull requests to us, and our code
review guideline.
 
## Author
 
Harry Liu - *Initial work* - [byliuyang](https://short-d.com/r/ghharry)
 
As the tech lead of Short, I am responsible for the overall planning, execution
and success of complex software solutions to meet users' needs.
 
I deeply believe in and am striving to achieve the right column of the
following diagram:
 
![Manager vs Leader](doc/leader-vs-manager.jpg)
 
## License
 
This project is maintained under MIT license