Skip to content

Commit 4278797

Browse files
jaehyeon-kimjaehyeon-kim
committed
fix merge conflict
2 parents 9c3cb96 + 282b598 commit 4278797

File tree

76 files changed

+6192
-251
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

76 files changed

+6192
-251
lines changed

‎.github/workflows/lint.yml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
name: lint
2+
3+
on: [push]
4+
5+
jobs:
6+
test-build:
7+
runs-on: ubuntu-latest
8+
steps:
9+
- uses: actions/checkout@v4
10+
- run: helm lint charts/*

‎.github/workflows/release.yml‎

Lines changed: 42 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,47 @@ jobs:
1919
git config user.name "$GITHUB_ACTOR"
2020
git config user.email "[email protected]"
2121
22-
- name: Run chart-releaser
23-
uses: helm/[email protected]
22+
- name: Install Helm
23+
uses: azure/setup-helm@5119fcb9089d432beecbf79bb2c7915207344b78 #v3.5
24+
with:
25+
version: v3.8.1
26+
27+
- name: Prepare GPG key # This step is for using exported keys and make your github runner
28+
run: |
29+
# Create a folder to store files
30+
gpg_dir=.cr-gpg
31+
mkdir "$gpg_dir"
32+
33+
# Refer keyring to private key of gpg
34+
keyring="$gpg_dir/secring.gpg"
35+
36+
# Store base64 GPG key into keyring
37+
base64 -d <<< "$GPG_KEYRING_BASE64" > "$keyring"
38+
39+
# Store passphrase data into a file
40+
passphrase_file="$gpg_dir/passphrase"
41+
echo "$GPG_PASSPHRASE" > "$passphrase_file"
42+
43+
# Save passphrase into github-environment
44+
echo "CR_PASSPHRASE_FILE=$passphrase_file" >> "$GITHUB_ENV"
45+
46+
# Save private key into github-environemnt
47+
echo "CR_KEYRING=$keyring" >> "$GITHUB_ENV"
48+
env:
49+
GPG_KEYRING_BASE64: "${{ secrets.GPG_KEYRING_BASE64 }}" # Refer secrets of github above
50+
GPG_PASSPHRASE: "${{ secrets.GPG_PASSPHRASE }}"
51+
52+
- name: Add repositories
53+
run: |
54+
for dir in $(ls -d charts/*/); do
55+
helm dependency list $dir 2> /dev/null | tail +2 | head -n -1 | awk '{ print "helm repo add " $1 " " $3 }' | while read cmd; do $cmd; done
56+
done
57+
58+
- name: Run chart-releaser #this is used to generate new version of helm chart along with some file with extension .prov
59+
uses: helm/chart-releaser-action@a917fd15b20e8b64b94d9158ad54cd6345335584 #v1.6.0
60+
with:
61+
skip_existing: true
2462
env:
2563
CR_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
64+
CR_KEY: "${{ secrets.CR_KEY }}" # Key name used while creating key
65+
CR_SIGN: true # Set to true to sign images

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,2 @@
11
.idea
2+
.vscode

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# Factor House Helm Charts
22

33
[![Release Charts](https://github.com/factorhouse/helm-charts/actions/workflows/release.yml/badge.svg?branch=main)](https://github.com/factorhouse/helm-charts/actions/workflows/release.yml)
4-
[![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/factorhouse)](https://artifacthub.io/packages/search?repo=factorhouse)
4+
[![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/factorhouse)](https://artifacthub.io/packages/search?org=factorhouse)
55

66
Official Helm Charts for Factor House products. Currently supported:
77

‎_readme_gen/README.md‎

Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,119 @@
1+
# Helm Chart README Generator
2+
3+
This directory contains the source code and automation script for generating the `README.md` files for our various Helm charts. The purpose of this tool is to solve the problem of keeping multiple, similar `README.md` files in sync.
4+
5+
## How It Works
6+
7+
The system is based on two key concepts: **Partials** and **Templates**.
8+
9+
- **Partials (`_partials/`):** These are small, reusable Markdown files that contain a single, specific piece of documentation, like installation instructions or prerequisites.
10+
- **Templates (`templates/`):** These are the master "blueprint" files for each final `README.md`. They define the structure of the document and use a special directive, `@@include(...)`, to pull in the content from the partials.
11+
12+
The `generate.py` script reads a template, finds all the `@@include` directives, replaces them with the content from the corresponding partial files, and writes the final, assembled `README.md` to the correct chart directory.
13+
14+
### Directory Structure
15+
16+
The project is organized by product to keep the source files separate and clean.
17+
18+
```
19+
_readme_gen/
20+
├── README.md # This file
21+
├── generate.py # The master Python script
22+
│
23+
├── flex/ # Source files for the "flex" product
24+
│ ├── _partials/ # Reusable Markdown snippets for flex
25+
│ │ └── _... .md
26+
│ └── templates/ # Master templates for each flex README
27+
│ └── flex.template.md
28+
│
29+
└── kpow/ # Source files for the "kpow" product
30+
├── _partials/ # Reusable Markdown snippets for kpow
31+
│ └── _... .md
32+
└── templates/ # Master templates for each kpow README
33+
└── kpow.template.md
34+
```
35+
36+
---
37+
38+
## Usage
39+
40+
The script can be run from the command line to generate READMEs for a specific product or for all products at once.
41+
42+
### Generating READMEs for a Specific Product
43+
44+
Provide the product name (`kpow`, `flex`, etc.) as an argument to the script. This is useful for local development when you are only working on one product.
45+
46+
**Syntax:**
47+
48+
```bash
49+
python generate.py <product_name>
50+
```
51+
52+
**Examples:**
53+
54+
```bash
55+
# Generate all READMEs for Kpow
56+
python generate.py kpow
57+
58+
# Generate all READMEs for Flex
59+
python generate.py flex
60+
```
61+
62+
### Generating All READMEs
63+
64+
Running the script without any arguments will default to generating the READMEs for **all** products defined in the configuration. This is the mode used by our CI/CD automation.
65+
66+
**Examples:**
67+
68+
```bash
69+
# Generate all READMEs for all products
70+
python generate.py
71+
```
72+
73+
Or explicitly:
74+
75+
```bash
76+
python generate.py all
77+
```
78+
79+
### Getting Help
80+
81+
To see the list of available products and options, use the `--help` flag.
82+
83+
```bash
84+
python generate.py --help
85+
```
86+
87+
---
88+
89+
## Adding a New Product
90+
91+
To add a new product (e.g., "platform") to the generator, follow these steps:
92+
93+
1. **Create the Directory Structure:**
94+
Create a new folder inside `_readme_gen/` with the product's name, and add `_partials` and `templates` subdirectories.
95+
96+
```
97+
_readme_gen/
98+
└── platform/
99+
├── _partials/
100+
└── templates/
101+
```
102+
103+
2. **Add Content:**
104+
Create your reusable `.md` files in `_partials/` and your master `platform.template.md` files in `templates/`.
105+
106+
3. **Update the Configuration:**
107+
Open `generate.py` and add a new top-level entry for `"platform"` inside the `README_CONFIG` dictionary. Follow the existing structure to define the template and output paths for each of your new chart's READMEs.
108+
109+
**Example addition to `README_CONFIG`:**
110+
111+
```python
112+
"gizmo": [
113+
{
114+
"template": BASE_DIR / "platform" / "templates/platform.template.md",
115+
"output": BASE_DIR / f"../{CHART_FOLDER}/gizmo/README.md",
116+
},
117+
# ... more chart variations for platform
118+
],
119+
```

‎_readme_gen/flex/_partials/_configure-eks-environment.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
### Configure Kubernetes/EKS
2+
3+
You need to connect to a Kubernetes environment before you can install Kpow.
4+
5+
The following examples demonstrate installing Kpow in [Amazon EKS](https://aws.amazon.com/eks/).
6+
7+
```bash
8+
aws eks --region <your-aws-region> update-kubeconfig --name <your-eks-cluster-name>
9+
10+
Updated context arn:aws:eks:<your-aws-region>:123123123:cluster/<your-eks-cluster-name> in /your/.kube/config
11+
```
12+
13+
#### Confirm Kubernetes Cluster Availability
14+
15+
```bash
16+
kubectl get nodes
17+
18+
NAME STATUS ROLES AGE VERSION
19+
ip-192-168-...-21.ec2.internal Ready <none> 2m15s v1.32.9-eks-113cf36
20+
...
21+
```

‎_readme_gen/flex/_partials/_configure-helm-flex.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
Add the Factor House Helm Repository in order to use the Flex Helm Chart.
2+
3+
```bash
4+
helm repo add factorhouse https://charts.factorhouse.io
5+
```
6+
7+
Update Helm repositories to ensure you install the latest version of Flex.
8+
9+
```bash
10+
helm repo update
11+
```

‎_readme_gen/flex/_partials/_feature-manage-flex-instance.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
#### Set the $POD_NAME variable and test the Flex UI
2+
3+
Follow the notes instructions to set the $POD_NAME variable and configure port forwarding to the Flex UI.
4+
5+
```bash
6+
export POD_NAME=$(kubectl get pods --namespace factorhouse -l "app.kubernetes.io/name=flex,app.kubernetes.io/instance=flex" -o jsonpath="{.items[0].metadata.name}")
7+
echo "Visit http://127.0.0.1:3000 to use your application"
8+
kubectl --namespace factorhouse port-forward $POD_NAME 3000:3000
9+
```
10+
11+
Flex is now available on [http://127.0.0.1:3000](http://127.0.0.1:3000).
12+
13+
#### Check the Flex Pod
14+
15+
```bash
16+
kubectl describe pods --namespace factorhouse
17+
18+
Name: flex-9988df6b6-vvf8z
19+
Namespace: factorhouse
20+
Priority: 0
21+
Node: ip-172-31-33-42.ap-southeast-2.compute.internal/172.31.33.42
22+
Start Time: Mon, 31 May 2021 17:22:22 +1000
23+
Labels: app.kubernetes.io/instance=flex
24+
app.kubernetes.io/name=flex
25+
pod-template-hash=9988df6b6
26+
Annotations: kubernetes.io/psp: eks.privileged
27+
Status: Running
28+
```
29+
30+
#### View the Flex Pod Logs
31+
32+
```bash
33+
kubectl logs --namespace factorhouse $POD_NAME
34+
35+
11:36:49.111 INFO [main] flex.system ? start Flex
36+
...
37+
```
38+
39+
#### Remove Flex
40+
41+
```bash
42+
helm delete --namespace factorhouse flex
43+
```

‎_readme_gen/flex/_partials/_feature-manage-sensitive-env-vars.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
This helm chart accepts the name of a secret containing sensitive parameters, e.g.
2+
3+
```yaml
4+
apiVersion: v1
5+
kind: Secret
6+
metadata:
7+
name: flex-secrets
8+
data:
9+
SASL_JAAS_CONFIG: a3JnLmFwYWNoXS5rYWZrYS5jb21tb24uc2VjdXJpdHkucGxhaW4uUGxhaW5Mb2dpbk2vZHVsZSByZXF1aXJiZCB1c2VybmFtZT0iTFQ1V0ZaV1BRWUpHNzRJQyIgcGFzc3dvcmQ9IjlYUFVYS3BLYUQxYzVJdXVNRjRPKzZ2NxJ0a1E4aS9yWUp6YlppdlgvZnNiTG51eGY4SnlFT1dUeXMvTnJ1bTAiBwo=
10+
CONFLUENT_API_SECRET: NFJSejlReFNTTXlTcGhXdjNLMHNYY1F6UGNURmdadlNYT0ZXSXViWFJySmx2N3A2WStSenROQnVpYThvNG1NSRo=
11+
```
12+
13+
```bash
14+
kubectl apply -f ./flex-secrets.yaml --namespace factorhouse
15+
```
16+
17+
Then run the helm chart (this can be used in conjunction with `envFromConfigMap`)
18+
19+
See the Kubernetes documentation
20+
on [configuring all key value pairs in a secret as environment variables](https://kubernetes.io/docs/tasks/inject-data-application/distribute-credentials-secure/#configure-all-key-value-pairs-in-a-secret-as-container-environment-variables)
21+
for more information.

‎_readme_gen/flex/_partials/_feature-memory-cpu-requirements.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
The chart runs Flex with Guaranteed QoS, having resource request and limit set to these values by default:
2+
3+
```yaml
4+
resources:
5+
limits:
6+
cpu: 2
7+
memory: 8Gi
8+
requests:
9+
cpu: 2
10+
memory: 8Gi
11+
```
12+
13+
These default resource settings are conservative, suited to a deployment of Flex that manages multiple Flink clusters
14+
and associated resources.
15+
16+
When running Flex with a single Flink cluster you can experiment with reducing those resources as far as our suggested
17+
minimum:
18+
19+
#### Minimum Resource Requirements
20+
21+
```yaml
22+
resources:
23+
limits:
24+
cpu: 1
25+
memory: 2Gi
26+
requests:
27+
cpu: 1
28+
memory: 2Gi
29+
```

0 commit comments

Comments
 (0)