Skip to content

Commit 6dfe002

Browse files
mikefarahmikefarah
committed
Updating contrib doc
1 parent ed4f468 commit 6dfe002

File tree

1 file changed

+189
-32
lines changed

1 file changed

+189
-32
lines changed

‎CONTRIBUTING.md‎

Lines changed: 189 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -3,55 +3,212 @@ Not all new PRs will be merged in
33

44
It's recommended to check with the owner first (e.g. raise an issue) to discuss a new feature before developing, to ensure your hard efforts don't go to waste.
55

6-
PRs to fix bugs and issues are almost always be welcome, just make sure you write tests as well.
6+
PRs to fix bugs and issues are almost always welcome :pray: please ensure you write tests as well.
77

8-
** PRs that significantly refactor code and release pipelines PRs will generally _NOT_ be accepted **
8+
The following types of PRs will _not_ be accepted:
9+
- **Significant refactors** take a lot of time to understand and can have all sorts of unintended side effects. If you think there's a better way to do things (that requires significant changes) raise an issue for discussion first :)
10+
- **Release pipeline PRs** are a security risk - it's too easy for a serious vulnerability to sneak in (either intended or not). If there is a new cool way of releasing things, raise an issue for discussion first - it will need to be gone over with a fine tooth comb.
11+
- **Version bumps** are handled by dependabot, the bot will auto-raise PRs and they will be regularly merged in.
12+
- **New release platforms** At this stage, yq is not going to maintain any other release platforms other than GitHub and Docker - that said, I'm more than happy to put in other community maintained methods in the README for visibility :heart:
913

10-
Significant refactors take a lot of time to understand and can have all sorts of unintended side effects.
1114

12-
Release pipeline PRs are a security risk - it's too easy for a serious vulnerability to sneak in (either intended or not). If there is a new cool way of releasing things, raise an issue for discussion first - it will need to be gone over with a fine tooth comb.
15+
# Development
1316

14-
At this stage, yq is not going to maintain any other release platforms other than GitHub and Docker - that said, I'm more than happy to put in other community maintained methods in the README for visibility :heart:
17+
## Initial Setup
1518

19+
1. Install [Golang](https://golang.org/) (version 1.24.0 or later)
20+
2. Run `scripts/devtools.sh` to install required development tools:
21+
- golangci-lint for code linting
22+
- gosec for security analysis
23+
3. Run `make [local] vendor` to install vendor dependencies
24+
4. Run `make [local] test` to ensure you can run the existing tests
1625

17-
# Development
26+
## Development Workflow
27+
28+
1. **Write unit tests first** - Changes will not be accepted without corresponding unit tests (see Testing section below)
29+
2. **Make your code changes**
30+
3. **Run tests and linting**: `make [local] test` (this runs formatting, linting, security checks, and tests)
31+
4. **Create your PR** and get kudos! :)
32+
33+
## Make Commands
34+
35+
- Use `make [local] <command>` for local development (runs in Docker container)
36+
- Use `make <command>` for CI/CD environments
37+
- Common commands:
38+
- `make [local] vendor` - Install dependencies
39+
- `make [local] test` - Run all checks and tests
40+
- `make [local] build` - Build the yq binary
41+
- `make [local] format` - Format code
42+
- `make [local] check` - Run linting and security checks
43+
44+
# Code Quality
45+
46+
## Linting and Formatting
47+
48+
The project uses strict linting rules defined in `.golangci.yml`. All code must pass:
49+
50+
- **Code formatting**: gofmt, goimports, gci
51+
- **Linting**: revive, errorlint, gosec, misspell, and others
52+
- **Security checks**: gosec security analysis
53+
- **Spelling checks**: misspell detection
54+
55+
Run `make [local] check` to verify your code meets all quality standards.
56+
57+
## Code Style Guidelines
58+
59+
- Follow standard Go conventions
60+
- Use meaningful variable names
61+
- Add comments for public functions and complex logic
62+
- Keep functions focused and reasonably sized
63+
- Use the project's existing patterns and conventions
64+
65+
# Testing
66+
67+
## Test Structure
68+
69+
Tests in yq use the `expressionScenario` pattern. Each test scenario includes:
70+
- `expression`: The yq expression to test
71+
- `document`: Input YAML/JSON (optional)
72+
- `expected`: Expected output
73+
- `skipDoc`: Whether to skip documentation generation
74+
75+
## Writing Tests
76+
77+
1. **Find the appropriate test file** (e.g., `operator_add_test.go` for addition operations)
78+
2. **Add your test scenario** to the `*OperatorScenarios` slice
79+
3. **Run the specific test**: `go test -run TestAddOperatorScenarios` (replace with appropriate test name)
80+
4. **Verify documentation generation** (see Documentation section)
81+
82+
## Test Examples
1883

19-
1. Install (Golang)[https://golang.org/]
20-
1. Run `scripts/devtools.sh` to install the required devtools
21-
2. Run `make [local] vendor` to install the vendor dependencies
22-
2. Run `make [local] test` to ensure you can run the existing tests
23-
3. Write unit tests - (see existing examples). Changes will not be accepted without corresponding unit tests.
24-
4. Make the code changes.
25-
5. `make [local] test` to lint code and run tests
26-
6. Profit! ok no profit, but raise a PR and get kudos :)
84+
```go
85+
var addOperatorScenarios = []expressionScenario{
86+
{
87+
skipDoc: true,
88+
expression: `"foo" + "bar"`,
89+
expected: []string{
90+
"D0, P[], (!!str)::foobar\n",
91+
},
92+
},
93+
{
94+
document: "apples: 3",
95+
expression: `.apples + 3`,
96+
expected: []string{
97+
"D0, P[apples], (!!int)::6\n",
98+
},
99+
},
100+
}
101+
```
27102

103+
## Running Tests
104+
105+
- **All tests**: `make [local] test`
106+
- **Specific test**: `go test -run TestName`
107+
- **With coverage**: `make [local] cover`
28108

29109
# Documentation
30110

31-
The various operator documentation (e.g. 'strings') are generated from the 'master' branch, and have a statically defined header (e.g. `pkg/yqlib/doc/operators/headers/add.md`) and the bulk of the docs are generated from the unit tests e.g. `pkg/yqlib/operator_add_test.go`.
111+
## Documentation Generation
112+
113+
The project uses a documentation system that combines static headers with dynamically generated content from tests.
114+
115+
### How It Works
116+
117+
1. **Static headers** are defined in `pkg/yqlib/doc/operators/headers/*.md`
118+
2. **Dynamic content** is generated from test scenarios in `*_test.go` files
119+
3. **Generated docs** are created in `pkg/yqlib/doc/*.md` by concatenating headers with test-generated content
120+
4. **Documentation is synced** to the gitbook branch for the website
121+
122+
### Updating Operator Documentation
123+
124+
#### For Test-Generated Documentation
125+
126+
Most operator documentation is generated from tests. To update:
127+
128+
1. **Find the test file** (e.g., `operator_add_test.go`)
129+
2. **Update test scenarios** - each `expressionScenario` with `skipDoc: false` becomes documentation
130+
3. **Run the test** to regenerate docs:
131+
```bash
132+
cd pkg/yqlib
133+
go test -run TestAddOperatorScenarios
134+
```
135+
4. **Verify the generated documentation** in `pkg/yqlib/doc/add.md`
136+
5. **Create a PR** with your changes
137+
138+
#### For Header-Only Documentation
139+
140+
If documentation exists only in `headers/*.md` files:
141+
1. **Update the header file directly** (e.g., `pkg/yqlib/doc/operators/headers/add.md`)
142+
2. **Create a PR** with your changes
143+
144+
### Updating Static Documentation
145+
146+
For documentation not in the master branch:
147+
148+
1. **Check the gitbook branch** for additional pages
149+
2. **Update the `*.md` files** directly
150+
3. **Create a PR** to the gitbook branch
151+
152+
### Documentation Best Practices
153+
154+
- **Write clear, concise examples** in test scenarios
155+
- **Use meaningful variable names** in examples
156+
- **Include edge cases** and error conditions
157+
- **Test your documentation changes** by running the specific test
158+
- **Verify generated output** matches expectations
159+
160+
Note: PRs with small changes (e.g. minor typos) may not be merged (see https://joel.net/how-one-guy-ruined-hacktoberfest2020-drama).
161+
162+
# Troubleshooting
32163

33-
The pipeline will run the tests and automatically concatenate the files together, and put them under
34-
`pkg/qylib/doc/add.md`. These files are checked in the master branch (and are copied to the gitbook branch as part of the release process).
164+
## Common Setup Issues
35165

166+
### Docker/Podman Issues
167+
- **Problem**: `make` commands fail with Docker errors
168+
- **Solution**: Ensure Docker or Podman is running and accessible
169+
- **Alternative**: Use `make local <command>` to run in containers
36170

37-
Remaining static documentation is in the 'githook' branch (where the generated docs are copied across into)
171+
### Go Version Issues
172+
- **Problem**: Build fails with Go version errors
173+
- **Solution**: Ensure you have Go 1.24.0 or later installed
174+
- **Check**: Run `go version` to verify
38175

39-
## How to contribute
176+
### Vendor Dependencies
177+
- **Problem**: `make vendor` fails or dependencies are outdated
178+
- **Solution**:
179+
```bash
180+
go mod tidy
181+
make [local] vendor
182+
```
40183

41-
The first step is to find if what you want is automatically generated or not - start by looking in the master branch.
184+
### Linting Failures
185+
- **Problem**: `make check` fails with linting errors
186+
- **Solution**:
187+
```bash
188+
make [local] format # Auto-fix formatting
189+
# Manually fix remaining linting issues
190+
make [local] check # Verify fixes
191+
```
42192

43-
Note that PRs with small changes (e.g. minor typos) may not be merged (see https://joel.net/how-one-guy-ruined-hacktoberfest2020-drama).
193+
### Test Failures
194+
- **Problem**: Tests fail locally but pass in CI
195+
- **Solution**:
196+
```bash
197+
make [local] test # Run in Docker container
198+
```
44199

45-
### Updating dynamic documentation from master
46-
- Search for the documentation you want to update. If you find matches in a `*_test.go` file - update that, as that will automatically update the matching `*.md` file
47-
- Assuming you are updating a `*_test.go` file, once updated, run the test to regenerated the docs. E.g. for the 'Add' test generated docs, from the pkg/yqlib folder run:
48-
`go test -run TestAddOperatorScenarios` which will run that test defined in the `operator_add_test.go` file.
49-
- Ensure the tests still pass, and check the generated documentation have your update.
50-
- Note: If the documentation is only in a `headers/*.md` file, then just update that directly
51-
- Raise a PR to merge the changes into master!
200+
### Documentation Generation Issues
201+
- **Problem**: Generated docs don't update after test changes
202+
- **Solution**:
203+
```bash
204+
cd pkg/yqlib
205+
go test -run TestSpecificOperatorScenarios
206+
# Check if generated file updated in pkg/yqlib/doc/
207+
```
52208

53-
### Updating static documentation from the gitbook branch
54-
If you haven't found what you want to update in the master branch, then check the gitbook branch directly as there are a few pages in there that are not in master.
209+
## Getting Help
55210

56-
- Update the `*.md` files
57-
- Raise a PR to merge the changes into gitbook.
211+
- **Check existing issues**: Search GitHub issues for similar problems
212+
- **Create an issue**: If you can't find a solution, create a detailed issue
213+
- **Ask questions**: Use GitHub Discussions for general questions
214+
- **Join the community**: Check the project's community channels

0 commit comments

Comments
 (0)