Compare commits

..

82 Commits

Author SHA1 Message Date
Volodymyr Zotov
54eed0c81c Merge pull request #217 from 1Password/release/v1.9.1
Prepare Release - v1.9.1
2025-07-14 10:47:55 -05:00
Volodymyr Zotov
8bd7d519fe Update changelog 2025-07-14 10:44:24 -05:00
Volodymyr Zotov
2823a571e9 Prepare release v1.9.1 2025-07-14 10:37:35 -05:00
Volodymyr Zotov
772e708f02 Merge pull request #212 from 1Password/vzt/fix-panic-when-reading-file
Fix panic when reading file
2025-07-14 10:30:39 -05:00
Volodymyr Zotov
1d613eac2b Fix logging errors so it won't panic 2025-07-10 15:46:02 -05:00
Volodymyr Zotov
dbd7408fac Update errors text 2025-07-10 15:40:53 -05:00
Volodymyr Zotov
6ef0da2d17 Add file from document 2025-07-10 14:03:57 -05:00
Volodymyr Zotov
b35acb7d13 Add test case for item with file 2025-07-08 16:57:59 -05:00
Volodymyr Zotov
9cee6595d5 Set file content after fetching it 2025-07-08 16:46:10 -05:00
Volodymyr Zotov
24d3f6f043 Retry when getting content file via Connect
Connect has a delay when synchronizing files and returns a 500 error in this case, this function implements a retry mechanism. In this case we handle retries in the code and do not print redundunt errors in the POD logs.
2025-07-08 16:44:09 -05:00
Volodymyr Zotov
5980e7e63a Wrap the error to not lose child errors 2025-07-08 16:42:38 -05:00
Volodymyr Zotov
1e9c04ee05 Fix logging the error so it doesn't panic 2025-07-08 16:41:49 -05:00
Eduard Filip
a5416f4532 Merge pull request #210 from 1Password/eddy/fix-dependabot-alerts
Address dependabot alerts
2025-07-08 19:13:56 +02:00
Eddy Filip
7e739a6fc7 Address dependabot alerts 2025-07-07 22:48:27 +02:00
Volodymyr Zotov
0f1dcdd38a Merge pull request #205 from 1Password/release/v1.9.0
Prepare Release - v1.9.0
2025-06-30 09:56:41 -05:00
Volodymyr Zotov
4c04c6699b Update changleog 2025-06-27 11:17:28 -05:00
Volodymyr Zotov
cd03176aae Prepare release v1.9.0 2025-06-26 19:07:49 -05:00
Volodymyr Zotov
f194485a1b Merge pull request #202 from 1Password/vzt/fix-context
Pass context given in `Reconcile` function down to Connect/SDK requests
2025-06-24 14:21:29 -05:00
Volodymyr Zotov
d1254b06e7 Merge pull request #198 from 1Password/vzt/service-accounts-support
Service accounts support
2025-06-24 14:21:09 -05:00
Volodymyr Zotov
7c84f9d3a4 Remove prerequisites section from USAGEGUIDE.md 2025-06-24 14:18:52 -05:00
Volodymyr Zotov
13abcb9c8f Merge branch 'vzt/service-accounts-support' into vzt/fix-context
# Conflicts:
#	USAGEGUIDE.md
#	cmd/main.go
#	pkg/onepassword/client/client.go
#	pkg/onepassword/items.go
2025-06-17 13:21:39 -05:00
Volodymyr Zotov
b717823fd0 Update examples section 2025-06-17 13:12:58 -05:00
Volodymyr Zotov
c9b969def1 Update comments 2025-06-17 11:27:55 -05:00
Volodymyr Zotov
fd92ef86ab Revert changelog and version change as will be done in release MR
Revert version change as will be dne in release MR
2025-06-17 11:26:04 -05:00
Volodymyr Zotov
842c8febdc Update comment 2025-06-17 11:20:01 -05:00
Volodymyr Zotov
49a5e93c44 Remove GO111MODULE=on as it set to 'on' by default from Go 1.16 2025-06-17 11:19:21 -05:00
Volodymyr Zotov
ac646ec56c Rename vaultIdentifier and itemIdentifier for more clarity 2025-06-17 11:16:57 -05:00
Volodymyr Zotov
458ed24ca3 Check second vault ID 2025-06-16 22:05:06 -05:00
Volodymyr Zotov
bb7b565457 As we check for vault name, we can't initialize the array of exact size as don't know how many items well have ther 2025-06-16 22:04:23 -05:00
Volodymyr Zotov
17d44d90bd Check for empty list 2025-06-16 21:56:51 -05:00
Volodymyr Zotov
0903bacfbd Check Create_At in the test 2025-06-16 21:55:32 -05:00
Volodymyr Zotov
32360d8a83 Remove todos from mocked methods 2025-06-16 21:53:46 -05:00
Volodymyr Zotov
2373fbc87f Updated mapping to be faster 2025-06-16 21:32:00 -05:00
Volodymyr Zotov
704116b855 Remove user agent from Connect config as it's set automatically when initializing Connect client 2025-06-16 21:10:33 -05:00
Volodymyr Zotov
55b5781d7a Pass logger to print what what type of client is used Connect or Service Account 2025-06-16 21:04:22 -05:00
Volodymyr Zotov
1aa27fdba0 Sort imports 2025-06-16 20:56:00 -05:00
Volodymyr Zotov
f164a93b72 Re-arrange env vars 2025-06-16 20:51:16 -05:00
Volodymyr Zotov
9d0736285f Fix type 2025-06-16 20:18:57 -05:00
Volodymyr Zotov
aa1b4ba857 Capitalize 1Password in error 2025-06-16 20:12:38 -05:00
Volodymyr Zotov
ae9b673f96 Remove commented code 2025-06-16 20:11:13 -05:00
Volodymyr Zotov
a0475d7170 Check created_at in the SDK mapper test 2025-06-16 20:09:35 -05:00
Volodymyr Zotov
922f3c8929 Map CreatedAt 2025-06-16 20:07:29 -05:00
Volodymyr Zotov
1fa5bccec2 Upse copy to copy tags 2025-06-16 20:03:36 -05:00
Volodymyr Zotov
cff4d194ba Update constructor function name 2025-06-16 19:45:23 -05:00
Volodymyr Zotov
475860a364 Make changelog description more detailed 2025-06-16 19:43:45 -05:00
Volodymyr Zotov
64aad3d573 Revert version change, as should be done in the release MR 2025-06-16 19:40:54 -05:00
Volodymyr Zotov
0582c2d6e1 Bump go version to 1.24 2025-06-16 19:40:53 -05:00
Volodymyr Zotov
d1be03edd0 Update USAGEGUIDE.md 2025-06-16 19:40:50 -05:00
Volodymyr Zotov
83b294690a Revert version change, as should be done in the release MR 2025-06-13 16:23:45 -05:00
Volodymyr Zotov
ef7361b3c1 Bump go version to 1.24 2025-06-13 16:22:18 -05:00
Volodymyr Zotov
04c1fc1236 Update USAGEGUIDE.md 2025-06-13 16:15:41 -05:00
Volodymyr Zotov
3723c962fe Merge branch 'vzt/service-accounts-support' into vzt/fix-context
# Conflicts:
#	internal/controller/deployment_controller.go
#	internal/controller/onepassworditem_controller.go
2025-06-12 15:08:35 -05:00
Volodymyr Zotov
4d2120061d Bump onepassword-sdk-version 2025-06-12 10:08:55 -05:00
Volodymyr Zotov
c95078c34c Retry after 15 minutes, if get rate-limit error
As currently service account are last for 1 hour https://developer.1password.com/docs/service-accounts/rate-limits/#hourly-limits
2025-06-09 16:04:23 -05:00
Volodymyr Zotov
4527336c37 Increase default container resources
To avoid operator crashing in ~1 min after re-trying unrecoverable error, for example service account rate limit
2025-06-09 16:01:35 -05:00
Volodymyr Zotov
0b6b07b867 Requeue after 1 munute if faced reate limit error from 1password 2025-06-08 12:48:55 -05:00
Volodymyr Zotov
4baad12e10 Add instructions how to use Operator with Service Accounts 2025-06-08 11:23:10 -05:00
Volodymyr Zotov
efbe96e93a Use global context 2025-06-08 10:28:15 -05:00
Volodymyr Zotov
ac06f8db13 Add more logs and fix params order 2025-06-06 16:12:25 -05:00
Volodymyr Zotov
72511ed687 Return error if both Connect and Service Account credentials are provided 2025-06-06 12:56:17 -05:00
Volodymyr Zotov
4757263c66 Wrap errors so it's clear either error is coming from SDK or Connect 2025-06-06 12:53:56 -05:00
Volodymyr Zotov
97e06e5c4d Bump version to 1.9.0 2025-05-30 16:10:25 -05:00
Volodymyr Zotov
a432b42814 Update Docker file to install dependencies and build 2025-05-30 14:52:41 -05:00
Volodymyr Zotov
f88ea6696b Update tests to use testify mock 2025-05-30 14:30:06 -05:00
Volodymyr Zotov
1498c223a5 Use 1Password Client to initialize operator either with Connect or Service Accounts 2025-05-29 17:23:49 -05:00
Volodymyr Zotov
432f2c6cf6 Add Client instance that utilizes either Connect or SDK 2025-05-29 16:06:55 -05:00
Volodymyr Zotov
a49c6ee045 Add SDK client wrapper 2025-05-29 16:06:02 -05:00
Volodymyr Zotov
8881782559 Create Connect client wrapper 2025-05-29 13:12:03 -05:00
Volodymyr Zotov
dcb5d5675a Add internal models
These internal models are introduced to reduce decoupling. The idea is to operate internal model within the project boundaries and convert to appropriate Connect or SDK models in the places where it's necessary.
2025-05-29 11:30:17 -05:00
Volodymyr Zotov
b567b99774 Merge pull request #196 from 1Password/vzt/remove-vendor
Remove vendor folder
2025-05-28 09:06:49 -05:00
Volodymyr Zotov
02c90d424b Remove vendor folder 2025-05-27 14:56:57 -05:00
Kevin Gottsman
4428515407 Update README.md to show the apiVersion (#194)
Typo which doesn't show the apiVersion resulting in an error
2025-02-05 13:55:51 +01:00
Eduard Filip
949a840779 Update bug bounty process (#193) 2024-12-12 18:16:35 +01:00
Andi Titu
ced45c33d4 Merge pull request #192 from 1Password/AndyTitu/cpu-resource-request
Use requests instead of limits for cpu resource
2024-10-09 11:52:11 +03:00
Andi Titu
4091f80351 Use 0.2 instead of 0.2m 2024-10-09 11:45:20 +03:00
Andi Titu
c94e7a5557 use 0.2 instead of 0.2m 2024-10-09 11:44:53 +03:00
Andi Titu
3fbd0b32cd Use requests instead of limits for cpu resource 2024-10-09 11:35:32 +03:00
Simon Barendse
2c55fbc5ed Merge pull request #189 from plttn/patch-1
Fix typo in readme
2024-08-01 17:13:19 +02:00
Jack Platten
fcb97e1482 Update README.md
fix typo
2024-06-11 13:23:05 -07:00
Eduard Filip
b3346cbc25 Run 1Password/check-signed-commits-action for PRs (#188)
Add the 1Password/check-signed-commits-action that will leave a handy comment if a PR contains commits that are not signed.
2024-05-28 19:36:34 +02:00
Eduard Filip
8c0f1a7837 Add CONTRIBUTING.md (#186)
This provides the necessary steps for external contributors to build, test and add features to the operator.
2024-03-25 15:58:16 +01:00
Eduard Filip
eda5612827 Upgrade Operator SDK to v1.34.1 and update dependencies (#185)
This does the following updates:

* Upgrade to Operator SDK v1.34.1. This fixes building multi-arch images from Makefile. Check this MR from operator-framework for details.
* Update Go dependencies. This addresses Dependabot alert ["Golang protojson.Unmarshal function infinite loop when unmarshaling certain forms of invalid JSON"](https://github.com/1Password/onepassword-operator/security/dependabot/13).
* Update versions of the GitHub Actions used in the pipelines.
* Update Kubernetes related tools (such as controller-tools version, and operator-sdk for ci pipelines)

By updating dependencies, the pipelines no longer fail due to a panic error when running `make test`.
2024-03-25 15:41:18 +01:00
4150 changed files with 2155 additions and 1474046 deletions

View File

@@ -1 +1 @@
1.8.1
1.9.1

View File

@@ -9,10 +9,10 @@ jobs:
- name: Set up Go 1.x
uses: actions/setup-go@v4
with:
go-version: ^1.20
go-version: ^1.21
- name: Check out code into the Go module directory
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Build
run: go build -v ./...

View File

@@ -0,0 +1,13 @@
name: Check signed commits in PR
on: pull_request_target
jobs:
build:
name: Check signed commits in PR
permissions:
contents: read
pull-requests: write
runs-on: ubuntu-latest
steps:
- name: Check signed commits in PR
uses: 1Password/check-signed-commits-action@v1

View File

@@ -17,7 +17,7 @@ jobs:
-
id: is_release_branch_without_pr
name: Find matching PR
uses: actions/github-script@v6
uses: actions/github-script@v7
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
@@ -43,7 +43,7 @@ jobs:
name: Create Release Pull Request
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Parse release version
id: get_version

View File

@@ -12,13 +12,13 @@ jobs:
DOCKER_CLI_EXPERIMENTAL: "enabled"
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Docker meta
id: meta
uses: docker/metadata-action@v4
uses: docker/metadata-action@v5
with:
images: |
1password/onepassword-operator
@@ -33,19 +33,19 @@ jobs:
run: echo ::set-output name=VERSION::${GITHUB_REF#refs/tags/v}
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
- name: Docker Login
uses: docker/login-action@v2
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push
uses: docker/build-push-action@v3
uses: docker/build-push-action@v5
with:
context: .
file: Dockerfile

View File

@@ -12,6 +12,30 @@
---
[//]: # (START/v1.9.1)
# v1.9.1
## Fixes
* Operator no longer panics when handling 1Password items containing files. {#209}
## Security
* HTTP Proxy bypass using IPv6 Zone IDs in golang.org/x/net. {#210}
* golang.org/x/net vulnerable to Cross-site Scripting. {#210}
---
[//]: # (START/v1.9.0)
# v1.9.0
## Features
* Enable the Operator to authenticate to 1Password using service accounts. {#160}
## Fixes
* Update Operator to use SDK v1.34.1. {#185}
* Pass Kubernetes context down to SDK/Connect. {#199}
---
[//]: # (START/v1.8.1)
# v1.8.1

72
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,72 @@
# Contributing
Thank you for your interest in contributing to the 1Password Kubernetes Operator project 👋! Before you start, please take a moment to read through this guide to understand our contribution process.
## Testing
- For functional testing, run the local version of the operator. From the project root:
```sh
# Go to the K8s environment (e.g. minikube)
eval $(minikube docker-env)
# Build the local Docker image for the operator
make docker-build
# Deploy the operator
make deploy
# Remove the operator from K8s
make undeploy
```
- For testing the changes made to the `OnePasswordItem` Custom Resource Definition (CRD), you need to re-generate the object:
```sh
make manifests
```
- Run tests for the operator:
```sh
make test
```
You can check other available commands that may come in handy by running `make help`.
## Debugging
- Running `kubectl describe pod` will fetch details about pods. This includes configuration information about the container(s) and Pod (labels, resource requirements, etc) and status information about the container(s) and Pod (state, readiness, restart count, events, etc.).
- Running `kubectl logs ${POD_NAME} ${CONTAINER_NAME}` will print the logs from the container(s) in a pod. This can help with debugging issues by inspection.
- Running `kubectl exec ${POD_NAME} -c ${CONTAINER_NAME} -- ${CMD}` allows executing a command inside a specific container.
For more debugging documentation, see: https://kubernetes.io/docs/tasks/debug/debug-application/debug-pods/
## Documentation Updates
If applicable, update the [USAGEGUIDE.md](./USAGEGUIDE.md) and [README.md](./README.md) to reflect any changes introduced by the new code.
## Sign your commits
To get your PR merged, we require you to sign your commits. There are three options you can choose from.
### Sign commits with 1Password
You can sign commits using 1Password, which lets you sign commits with biometrics without the signing key leaving the local 1Password process.
Learn how to use [1Password to sign your commits](https://developer.1password.com/docs/ssh/git-commit-signing/).
### Sign commits with ssh-agent
Follow the steps below to set up commit signing with `ssh-agent`:
1. [Generate an SSH key and add it to ssh-agent](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent)
2. [Add the SSH key to your GitHub account](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account)
3. [Configure git to use your SSH key for commits signing](https://docs.github.com/en/authentication/managing-commit-signature-verification/telling-git-about-your-signing-key#telling-git-about-your-ssh-key)
### Sign commits with gpg
Follow the steps below to set up commit signing with `gpg`:
1. [Generate a GPG key](https://docs.github.com/en/authentication/managing-commit-signature-verification/generating-a-new-gpg-key)
2. [Add the GPG key to your GitHub account](https://docs.github.com/en/authentication/managing-commit-signature-verification/adding-a-gpg-key-to-your-github-account)
3. [Configure git to use your GPG key for commits signing](https://docs.github.com/en/authentication/managing-commit-signature-verification/telling-git-about-your-signing-key#telling-git-about-your-gpg-key)

View File

@@ -1,5 +1,5 @@
# Build the manager binary
FROM golang:1.21 as builder
FROM golang:1.24 as builder
ARG TARGETOS
ARG TARGETARCH
@@ -8,13 +8,15 @@ WORKDIR /workspace
COPY go.mod go.mod
COPY go.sum go.sum
# Download dependencies
RUN go mod download
# Copy the go source
COPY cmd/main.go cmd/main.go
COPY api/ api/
COPY internal/controller/ internal/controller/
COPY pkg/ pkg/
COPY version/ version/
COPY vendor/ vendor/
# Build
# the GOARCH has not a default value to allow the binary be built according to the host where the command
@@ -25,7 +27,6 @@ RUN CGO_ENABLED=0 \
GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH} \
go build \
-ldflags "-X \"github.com/1Password/onepassword-operator/version.Version=$operator_version\"" \
-mod vendor \
-a -o manager cmd/main.go
# Use distroless as minimal base image to package the manager binary

View File

@@ -5,7 +5,7 @@ export MAIN_BRANCH ?= main
# To re-generate a bundle for another specific version without changing the standard setup, you can:
# - use the VERSION as arg of the bundle target (e.g make bundle VERSION=0.0.2)
# - use environment variables to overwrite this value (e.g export VERSION=0.0.2)
VERSION ?= 1.8.1
VERSION ?= 1.9.1
# CHANNELS define the bundle channels used in the bundle.
# Add a new line here if you would like to change its default config. (E.g CHANNELS = "candidate,fast,stable")
@@ -50,12 +50,12 @@ endif
# Set the Operator SDK version to use. By default, what is installed on the system is used.
# This is useful for CI or a project to utilize a specific version of the operator-sdk toolkit.
OPERATOR_SDK_VERSION ?= v1.33.0
OPERATOR_SDK_VERSION ?= v1.34.1
# Image URL to use all building/pushing image targets
IMG ?= 1password/onepassword-operator:latest
# ENVTEST_K8S_VERSION refers to the version of kubebuilder assets to be downloaded by envtest binary.
ENVTEST_K8S_VERSION = 1.28.3
ENVTEST_K8S_VERSION = 1.29.1
# Get the currently used golang install path (in GOPATH/bin, unless GOBIN is set)
ifeq (,$(shell go env GOBIN))
@@ -146,14 +146,11 @@ docker-push: ## Push docker image with the manager.
# To properly provided solutions that supports more than one platform you should use this option.
PLATFORMS ?= linux/arm64,linux/amd64,linux/s390x,linux/ppc64le
.PHONY: docker-buildx
docker-buildx: test ## Build and push docker image for the manager for cross-platform support
# copy existing Dockerfile and insert --platform=${BUILDPLATFORM} into Dockerfile.cross, and preserve the original Dockerfile
sed -e '1 s/\(^FROM\)/FROM --platform=\$$\{BUILDPLATFORM\}/; t' -e ' 1,// s//FROM --platform=\$$\{BUILDPLATFORM\}/' Dockerfile > Dockerfile.cross
docker-buildx: ## Build and push docker image for the manager for cross-platform support
- $(CONTAINER_TOOL) buildx create --name project-v3-builder
$(CONTAINER_TOOL) buildx use project-v3-builder
- $(CONTAINER_TOOL) buildx build --push --platform=$(PLATFORMS) --tag ${IMG} -f Dockerfile.cross .
- $(CONTAINER_TOOL) buildx build --push --platform=$(PLATFORMS) --tag ${IMG} -f Dockerfile .
- $(CONTAINER_TOOL) buildx rm project-v3-builder
rm Dockerfile.cross
##@ Deployment
@@ -197,7 +194,7 @@ ENVTEST ?= $(LOCALBIN)/setup-envtest
## Tool Versions
KUSTOMIZE_VERSION ?= v5.3.0
CONTROLLER_TOOLS_VERSION ?= v0.13.0
CONTROLLER_TOOLS_VERSION ?= v0.14.0
.PHONY: kustomize
kustomize: $(KUSTOMIZE) ## Download kustomize locally if necessary. If wrong version is installed, it will be removed before downloading.

View File

@@ -17,25 +17,31 @@ The 1Password Connect Kubernetes Operator provides the ability to integrate Kube
### 🚀 Quickstart
1. Add the [1Passsword Helm Chart](https://github.com/1Password/connect-helm-charts) to your repository.
1. Add the [1Password Helm Chart](https://github.com/1Password/connect-helm-charts) to your repository.
2. Run the following command to install Connect and the 1Password Kubernetes Operator in your infrastructure:
```
helm install connect 1password/connect --set-file connect.credentials=1password-credentials-demo.json --set operator.create=true --set operator.token.value = <your connect token>
```
3. Create a Kubernetes Secret from a 1Password item:
```apiVersion: onepassword.com/v1
```
apiVersion: onepassword.com/v1
kind: OnePasswordItem
metadata:
name: <item_name> #this name will also be used for naming the generated kubernetes secret
spec:
itemPath: "vaults/<vault_id_or_title>/items/<item_id_or_title>"
```
Deploy the OnePasswordItem to Kubernetes:
```
kubectl apply -f <your_item>.yaml
```
Check that the Kubernetes Secret has been generated:
```
@@ -43,6 +49,7 @@ kubectl get secret <secret_name>
```
### 📄 Usage
Refer to the [Usage Guide](USAGEGUIDE.md) for documentation on how to deploy and use the 1Password Operator.
## 💙 Community & Support
@@ -55,6 +62,4 @@ Refer to the [Usage Guide](USAGEGUIDE.md) for documentation on how to deploy and
1Password requests you practice responsible disclosure if you discover a vulnerability.
Please file requests via [**BugCrowd**](https://bugcrowd.com/agilebits).
For information about security practices, please visit the [1Password Bug Bounty Program](https://bugcrowd.com/agilebits).
Please file requests by sending an email to bugbounty@agilebits.com.

View File

@@ -5,107 +5,53 @@
## Table of Contents
- [Prerequisites](#prerequisites)
- [Deploying 1Password Connect to Kubernetes](#deploying-1password-connect-to-kubernetes)
- [Kubernetes Operator Deployment](#kubernetes-operator-deployment)
- [Usage](#usage)
- [Configuring Automatic Rolling Restarts of Deployments](#configuring-automatic-rolling-restarts-of-deployments)
- [Development](#development)
1. [Configuration Options](#configuration-options)
2. [Use Kubernetes Operator with Service Account](#use-kubernetes-operator-with-service-account)
- [Create a Service Account](#1-create-a-service-account)
- [Create a Kubernetes secret](#2-create-a-kubernetes-secret-for-the-service-account)
- [Deploy the Operator](#3-deploy-the-operator)
3. [Use Kubernetes Operator with Connect](#use-kubernetes-operator-with-connect)
- [Deploy with Helm](#1-deploy-with-helm)
- [Deploy manually](#2-deploy-manually)
4. [Logging level](#logging-level)
5. [Usage examples](#usage-examples)
6. [How 1Password Items Map to Kubernetes Secrets](#how-1password-items-map-to-kubernetes-secrets)
7. [Configuring Automatic Rolling Restarts of Deployments](#configuring-automatic-rolling-restarts-of-deployments)
8. [Development](#development)
## Prerequisites
- [1Password Command Line Tool Installed](https://1password.com/downloads/command-line/)
- [`kubectl` installed](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
- [`docker` installed](https://docs.docker.com/get-docker/)
- [A `1password-credentials.json` file generated and a 1Password Connect API Token issued for the K8s Operator integration](https://developer.1password.com/docs/connect/get-started/#step-1-set-up-a-secrets-automation-workflow)
---
## Deploying 1Password Connect to Kubernetes
## Configuration options
There are 2 ways 1Password Operator can talk to 1Password servers:
- [1Password Service Accounts](https://developer.1password.com/docs/service-accounts)
- [1Password Connect](https://developer.1password.com/docs/connect/)
If 1Password Connect is already running, you can skip this step.
---
There are options to deploy 1Password Connect:
## Use Kubernetes Operator with Service Account
- [Deploy with Helm](#deploy-with-helm)
- [Deploy using the Connect Operator](#deploy-using-the-connect-operator)
### Deploy with Helm
The 1Password Connect Helm Chart helps to simplify the deployment of 1Password Connect and the 1Password Connect Kubernetes Operator to Kubernetes.
[The 1Password Connect Helm Chart can be found here.](https://github.com/1Password/connect-helm-charts)
### Deploy using the Connect Operator
This guide will provide a quickstart option for deploying a default configuration of 1Password Connect via starting the deploying the 1Password Connect Operator, however, it is recommended that you instead deploy your own manifest file if customization of the 1Password Connect deployment is desired.
Encode the `1password-credentials.json` file you generated in the prerequisite steps and save it to a file named `op-session`:
### 1. [Create a service account](https://developer.1password.com/docs/service-accounts/get-started#create-a-service-account)
### 2. Create a Kubernetes secret for the Service Account
- Set `OP_SERVICE_ACCOUNT_TOKEN` environment variable to the service account token you created in the previous step. This token will be used by the operator to access 1Password items.
- Create Kubernetes secret:
```bash
cat 1password-credentials.json | base64 | \
tr '/+' '_-' | tr -d '=' | tr -d '\n' > op-session
kubectl create secret generic onepassword-service-account-token --from-literal=token="$OP_SERVICE_ACCOUNT_TOKEN"
```
Create a Kubernetes secret from the op-session file:
```bash
kubectl create secret generic op-credentials --from-file=op-session
```
Add the following environment variable to the onepassword-connect-operator container in `/config/manager/manager.yaml`:
```yaml
- name: MANAGE_CONNECT
value: "true"
```
Adding this environment variable will have the operator automatically deploy a default configuration of 1Password Connect to the current namespace.
### Kubernetes Operator Deployment
#### Create Kubernetes Secret for OP_CONNECT_TOKEN ####
Create a Connect token for the operator and save it as a Kubernetes Secret:
```bash
kubectl create secret generic onepassword-token --from-literal=token="<OP_CONNECT_TOKEN>"
```
If you do not have a token for the operator, you can generate a token and save it to Kubernetes with the following command:
```bash
kubectl create secret generic onepassword-token --from-literal=token=$(op create connect token <server> op-k8s-operator --vault <vault>)
```
**Deploying the Operator**
### 3. Deploy the Operator
An sample Deployment yaml can be found at `/config/manager/manager.yaml`.
To use Operator with Service Account, you need to set the `OP_SERVICE_ACCOUNT_TOKEN` environment variable in the `/config/manager/manager.yaml`. And remove `OP_CONNECT_TOKEN` and `OP_CONNECT_HOST` environment variables.
To further configure the 1Password Kubernetes Operator the following Environment variables can be set in the operator yaml:
- **OP_CONNECT_HOST** *(required)*: Specifies the host name within Kubernetes in which to access the 1Password Connect.
- **OP_SERVICE_ACCOUNT_TOKEN** *(required)*: Specifies Service Account token within Kubernetes to access the 1Password items.
- **WATCH_NAMESPACE:** *(default: watch all namespaces)*: Comma separated list of what Namespaces to watch for changes.
- **POLLING_INTERVAL** *(default: 600)*: The number of seconds the 1Password Kubernetes Operator will wait before checking for updates from 1Password Connect.
- **MANAGE_CONNECT** *(default: false)*: If set to true, on deployment of the operator, a default configuration of the OnePassword Connect Service will be deployed to the current namespace.
- **AUTO_RESTART** (default: false): If set to true, the operator will restart any deployment using a secret from 1Password Connect. This can be overwritten by namespace, deployment, or individual secret. More details on AUTO_RESTART can be found in the ["Configuring Automatic Rolling Restarts of Deployments"](#configuring-automatic-rolling-restarts-of-deployments) section.
- **POLLING_INTERVAL** *(default: 600)*: The number of seconds the 1Password Kubernetes Operator will wait before checking for updates from 1Password.
- **AUTO_RESTART** (default: false): If set to true, the operator will restart any deployment using a secret from 1Password. This can be overwritten by namespace, deployment, or individual secret. More details on AUTO_RESTART can be found in the ["Configuring Automatic Rolling Restarts of Deployments"](#configuring-automatic-rolling-restarts-of-deployments) section.
You can also set the logging level by setting `--zap-log-level` as an arg on the containers to either `debug`, `info` or `error`. (Note: the default value is `debug`.)
Example:
```yaml
.
.
.
containers:
- command:
- /manager
args:
- --leader-elect
- --zap-log-level=info
image: 1password/onepassword-operator:latest
.
.
.
```
To deploy the operator, simply run the following command:
```shell
@@ -118,59 +64,57 @@ make deploy
make undeploy
```
## Usage
---
To create a Kubernetes Secret from a 1Password item, create a yaml file with the following
## Use Kubernetes Operator with Connect
### 1. [Deploy with Helm](https://developer.1password.com/docs/k8s/operator/?deployment-type=helm#helm-step-1)
### 2. [Deploy manually](https://developer.1password.com/docs/k8s/operator/?deployment-type=manual#manual-step-1)
To further configure the 1Password Kubernetes Operator the following Environment variables can be set in the operator yaml:
- **OP_CONNECT_HOST** *(required)*: Specifies the host name within Kubernetes in which to access the 1Password Connect.
- **WATCH_NAMESPACE:** *(default: watch all namespaces)*: Comma separated list of what Namespaces to watch for changes.
- **POLLING_INTERVAL** *(default: 600)*: The number of seconds the 1Password Kubernetes Operator will wait before checking for updates from 1Password Connect.
- **MANAGE_CONNECT** *(default: false)*: If set to true, on deployment of the operator, a default configuration of the OnePassword Connect Service will be deployed to the current namespace.
- **AUTO_RESTART** (default: false): If set to true, the operator will restart any deployment using a secret from 1Password Connect. This can be overwritten by namespace, deployment, or individual secret. More details on AUTO_RESTART can be found in the ["Configuring Automatic Rolling Restarts of Deployments"](#configuring-automatic-rolling-restarts-of-deployments) section.
---
## Logging level
You can set the logging level by setting `--zap-log-level` as an arg on the containers to either `debug`, `info` or `error`. The default value is `debug`.
Example:
```yaml
apiVersion: onepassword.com/v1
kind: OnePasswordItem
metadata:
name: <item_name> #this name will also be used for naming the generated kubernetes secret
spec:
itemPath: "vaults/<vault_id_or_title>/items/<item_id_or_title>"
....
containers:
- command:
- /manager
args:
- --leader-elect
- --zap-log-level=info
image: 1password/onepassword-operator:latest
....
```
Deploy the OnePasswordItem to Kubernetes:
---
```bash
kubectl apply -f <your_item>.yaml
```
## Usage examples
Find usage [examples](https://developer.1password.com/docs/k8s/operator/?deployment-type=manual#usage-examples) on 1Password developer documentation.
To test that the Kubernetes Secret check that the following command returns a secret:
---
```bash
kubectl get secret <secret_name>
```
**Note:** Deleting the `OnePasswordItem` that you've created will automatically delete the created Kubernetes Secret.
To create a single Kubernetes Secret for a deployment, add the following annotations to the deployment metadata:
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: deployment-example
annotations:
operator.1password.io/item-path: "vaults/<vault_id_or_title>/items/<item_id_or_title>"
operator.1password.io/item-name: "<secret_name>"
```
Applying this yaml file will create a Kubernetes Secret with the name `<secret_name>` and contents from the location specified at the specified Item Path.
## How 1Password Items Map to Kubernetes Secrets
The contents of the Kubernetes secret will be key-value pairs in which the keys are the fields of the 1Password item and the values are the corresponding values stored in 1Password.
In case of fields that store files, the file's contents will be used as the value.
Within an item, if both a field storing a file and a field of another type have the same name, the file field will be ignored and the other field will take precedence.
**Note:** Deleting the Deployment that you've created will automatically delete the created Kubernetes Secret only if the deployment is still annotated with `operator.1password.io/item-path` and `operator.1password.io/item-name` and no other deployment is using the secret.
Deleting the Deployment that you've created will automatically delete the created Kubernetes Secret only if the deployment is still annotated with `operator.1password.io/item-path` and `operator.1password.io/item-name` and no other deployment is using the secret.
If a 1Password Item that is linked to a Kubernetes Secret is updated within the POLLING_INTERVAL the associated Kubernetes Secret will be updated. However, if you do not want a specific secret to be updated you can add the tag `operator.1password.io:ignore-secret` to the item stored in 1Password. While this tag is in place, any updates made to an item will not trigger an update to the associated secret in Kubernetes.
---
**NOTE**
If multiple 1Password vaults/items have the same `title` when using a title in the access path, the desired action will be performed on the oldest vault/item.
@@ -237,6 +181,8 @@ metadata:
If the value is not set, the auto restart settings on the deployment will be used.
---
## Development
### How it works

View File

@@ -25,6 +25,7 @@ SOFTWARE.
package main
import (
"context"
"errors"
"flag"
"fmt"
@@ -35,8 +36,6 @@ import (
"strings"
"time"
"github.com/1Password/connect-sdk-go/connect"
// Import all Kubernetes client auth plugins (e.g. Azure, GCP, OIDC, etc.)
// to ensure that exec-entrypoint and run can make use of them.
_ "k8s.io/client-go/plugin/pkg/client/auth"
@@ -54,6 +53,7 @@ import (
onepasswordcomv1 "github.com/1Password/onepassword-operator/api/v1"
"github.com/1Password/onepassword-operator/internal/controller"
op "github.com/1Password/onepassword-operator/pkg/onepassword"
opclient "github.com/1Password/onepassword-operator/pkg/onepassword/client"
"github.com/1Password/onepassword-operator/pkg/utils"
"github.com/1Password/onepassword-operator/version"
//+kubebuilder:scaffold:imports
@@ -113,6 +113,9 @@ func main() {
printVersion()
// Create a root context that will be cancelled on termination signals
ctx := ctrl.SetupSignalHandler()
watchNamespace, err := getWatchNamespace()
if err != nil {
setupLog.Error(err, "unable to get WatchNamespace, "+
@@ -153,16 +156,19 @@ func main() {
}
// Setup One Password Client
opConnectClient, err := connect.NewClientFromEnvironment()
opClient, err := opclient.NewFromEnvironment(ctx, opclient.Config{
Logger: setupLog,
Version: version.OperatorVersion,
})
if err != nil {
setupLog.Error(err, "unable to create Connect client")
setupLog.Error(err, "unable to create 1Password client")
os.Exit(1)
}
if err = (&controller.OnePasswordItemReconciler{
Client: mgr.GetClient(),
Scheme: mgr.GetScheme(),
OpConnectClient: opConnectClient,
Client: mgr.GetClient(),
Scheme: mgr.GetScheme(),
OpClient: opClient,
}).SetupWithManager(mgr); err != nil {
setupLog.Error(err, "unable to create controller", "controller", "OnePasswordItem")
os.Exit(1)
@@ -172,7 +178,7 @@ func main() {
if err = (&controller.DeploymentReconciler{
Client: mgr.GetClient(),
Scheme: mgr.GetScheme(),
OpConnectClient: opConnectClient,
OpClient: opClient,
OpAnnotationRegExp: r,
}).SetupWithManager(mgr); err != nil {
setupLog.Error(err, "unable to create controller", "controller", "Deployment")
@@ -183,10 +189,10 @@ func main() {
//Setup 1PasswordConnect
if shouldManageConnect() {
setupLog.Info("Automated Connect Management Enabled")
go func() {
go func(ctx context.Context) {
connectStarted := false
for connectStarted == false {
err := op.SetupConnect(mgr.GetClient(), deploymentNamespace)
err := op.SetupConnect(ctx, mgr.GetClient(), deploymentNamespace)
// Cache Not Started is an acceptable error. Retry until cache is started.
if err != nil && !errors.Is(err, &cache.ErrCacheNotStarted{}) {
setupLog.Error(err, "")
@@ -196,29 +202,29 @@ func main() {
connectStarted = true
}
}
}()
}(ctx)
} else {
setupLog.Info("Automated Connect Management Disabled")
}
// Setup update secrets task
updatedSecretsPoller := op.NewManager(mgr.GetClient(), opConnectClient, shouldAutoRestartDeployments())
updatedSecretsPoller := op.NewManager(mgr.GetClient(), opClient, shouldAutoRestartDeployments())
done := make(chan bool)
ticker := time.NewTicker(getPollingIntervalForUpdatingSecrets())
go func() {
go func(ctx context.Context) {
for {
select {
case <-done:
ticker.Stop()
return
case <-ticker.C:
err := updatedSecretsPoller.UpdateKubernetesSecretsTask()
err := updatedSecretsPoller.UpdateKubernetesSecretsTask(ctx)
if err != nil {
setupLog.Error(err, "error running update kubernetes secret task")
}
}
}
}()
}(ctx)
if err := mgr.AddHealthzCheck("healthz", healthz.Ping); err != nil {
setupLog.Error(err, "unable to set up health check")
@@ -230,7 +236,7 @@ func main() {
}
setupLog.Info("starting manager")
if err := mgr.Start(ctrl.SetupSignalHandler()); err != nil {
if err := mgr.Start(ctx); err != nil {
setupLog.Error(err, "problem running manager")
os.Exit(1)
}

View File

@@ -39,6 +39,7 @@ spec:
resources:
limits:
memory: "128Mi"
requests:
cpu: "0.2"
ports:
- containerPort: 8080
@@ -58,6 +59,7 @@ spec:
resources:
limits:
memory: "128Mi"
requests:
cpu: "0.2"
ports:
- containerPort: 8081

View File

@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.13.0
controller-gen.kubebuilder.io/version: v0.14.0
name: onepassworditems.onepassword.com
spec:
group: onepassword.com
@@ -20,14 +20,19 @@ spec:
description: OnePasswordItem is the Schema for the onepassworditems API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object

View File

@@ -75,25 +75,33 @@ spec:
image: 1password/onepassword-operator:latest
name: manager
env:
- name: WATCH_NAMESPACE
value: "default"
- name: OPERATOR_NAME
value: "onepassword-connect-operator"
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: OPERATOR_NAME
value: "onepassword-connect-operator"
- name: OP_CONNECT_HOST
value: "http://onepassword-connect:8080"
- name: WATCH_NAMESPACE
value: "default"
- name: POLLING_INTERVAL
value: "10"
- name: AUTO_RESTART
value: "false"
- name: OP_CONNECT_HOST
value: "http://onepassword-connect:8080"
- name: OP_CONNECT_TOKEN
valueFrom:
secretKeyRef:
name: onepassword-token
key: token
- name: AUTO_RESTART
- name: MANAGE_CONNECT
value: "false"
# Uncomment the following lines to enable service account token and comment out the OP_CONNECT_TOKEN, OP_CONNECT_HOST and MANAGE_CONNECT env vars.
# - name: OP_SERVICE_ACCOUNT_TOKEN
# valueFrom:
# secretKeyRef:
# name: onepassword-service-account-token
# key: token
securityContext:
allowPrivilegeEscalation: false
capabilities:
@@ -116,9 +124,9 @@ spec:
resources:
limits:
cpu: 500m
memory: 128Mi
memory: 512Mi
requests:
cpu: 10m
memory: 64Mi
cpu: 100m
memory: 128Mi
serviceAccountName: onepassword-connect-operator
terminationGracePeriodSeconds: 10

91
go.mod
View File

@@ -1,84 +1,91 @@
module github.com/1Password/onepassword-operator
go 1.21
go 1.24
toolchain go1.21.5
toolchain go1.24.4
require (
github.com/1Password/connect-sdk-go v1.5.3
github.com/onsi/ginkgo/v2 v2.13.2
github.com/1password/onepassword-sdk-go v0.3.1
github.com/go-logr/logr v1.4.2
github.com/onsi/ginkgo/v2 v2.14.0
github.com/onsi/gomega v1.30.0
github.com/stretchr/testify v1.8.4
k8s.io/api v0.29.0
k8s.io/apimachinery v0.29.0
k8s.io/client-go v0.29.0
github.com/stretchr/testify v1.10.0
k8s.io/api v0.29.3
k8s.io/apimachinery v0.29.3
k8s.io/client-go v0.29.3
k8s.io/kubectl v0.29.0
sigs.k8s.io/controller-runtime v0.16.3
sigs.k8s.io/controller-runtime v0.17.2
)
require (
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/emicklei/go-restful/v3 v3.11.0 // indirect
github.com/dylibso/observe-sdk/go v0.0.0-20240819160327-2d926c5d788a // indirect
github.com/emicklei/go-restful/v3 v3.12.0 // indirect
github.com/evanphx/json-patch v5.6.0+incompatible // indirect
github.com/evanphx/json-patch/v5 v5.6.0 // indirect
github.com/fsnotify/fsnotify v1.6.0 // indirect
github.com/go-logr/logr v1.3.0 // indirect
github.com/go-logr/zapr v1.2.4 // indirect
github.com/go-openapi/jsonpointer v0.19.6 // indirect
github.com/go-openapi/jsonreference v0.20.2 // indirect
github.com/go-openapi/swag v0.22.3 // indirect
github.com/evanphx/json-patch/v5 v5.9.0 // indirect
github.com/extism/go-sdk v1.7.0 // indirect
github.com/fsnotify/fsnotify v1.7.0 // indirect
github.com/go-logr/zapr v1.3.0 // indirect
github.com/go-openapi/jsonpointer v0.21.0 // indirect
github.com/go-openapi/jsonreference v0.21.0 // indirect
github.com/go-openapi/swag v0.23.0 // indirect
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 // indirect
github.com/gobwas/glob v0.2.3 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/gnostic-models v0.6.8 // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1 // indirect
github.com/google/uuid v1.3.0 // indirect
github.com/imdario/mergo v0.3.6 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/ianlancetaylor/demangle v0.0.0-20240805132620-81f5be970eca // indirect
github.com/imdario/mergo v0.3.16 // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/opentracing/opentracing-go v1.2.0 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/prometheus/client_golang v1.16.0 // indirect
github.com/prometheus/client_model v0.4.0 // indirect
github.com/prometheus/common v0.44.0 // indirect
github.com/prometheus/procfs v0.10.1 // indirect
github.com/prometheus/client_golang v1.19.0 // indirect
github.com/prometheus/client_model v0.6.0 // indirect
github.com/prometheus/common v0.51.1 // indirect
github.com/prometheus/procfs v0.13.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/stretchr/objx v0.5.2 // indirect
github.com/tetratelabs/wabin v0.0.0-20230304001439-f6f874872834 // indirect
github.com/tetratelabs/wazero v1.9.0 // indirect
github.com/uber/jaeger-client-go v2.30.0+incompatible // indirect
github.com/uber/jaeger-lib v2.4.1+incompatible // indirect
go.uber.org/atomic v1.10.0 // indirect
go.opentelemetry.io/proto/otlp v1.3.1 // indirect
go.uber.org/atomic v1.11.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.25.0 // indirect
golang.org/x/exp v0.0.0-20220722155223-a9213eeb770e // indirect
golang.org/x/net v0.17.0 // indirect
golang.org/x/oauth2 v0.10.0 // indirect
golang.org/x/sys v0.14.0 // indirect
golang.org/x/term v0.13.0 // indirect
golang.org/x/text v0.13.0 // indirect
golang.org/x/time v0.3.0 // indirect
golang.org/x/tools v0.14.0 // indirect
go.uber.org/zap v1.27.0 // indirect
golang.org/x/exp v0.0.0-20240318143956-a85f2c67cd81 // indirect
golang.org/x/net v0.41.0 // indirect
golang.org/x/oauth2 v0.30.0 // indirect
golang.org/x/sys v0.33.0 // indirect
golang.org/x/term v0.32.0 // indirect
golang.org/x/text v0.26.0 // indirect
golang.org/x/time v0.5.0 // indirect
golang.org/x/tools v0.33.0 // indirect
gomodules.xyz/jsonpatch/v2 v2.4.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/protobuf v1.31.0 // indirect
google.golang.org/protobuf v1.34.2 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/apiextensions-apiserver v0.28.3 // indirect
k8s.io/component-base v0.29.0 // indirect
k8s.io/klog/v2 v2.110.1 // indirect
k8s.io/kube-openapi v0.0.0-20231010175941-2dd684a91f00 // indirect
k8s.io/utils v0.0.0-20230726121419-3b25d923346b // indirect
k8s.io/apiextensions-apiserver v0.29.3 // indirect
k8s.io/component-base v0.29.3 // indirect
k8s.io/klog/v2 v2.120.1 // indirect
k8s.io/kube-openapi v0.0.0-20240322212309-b815d8309940 // indirect
k8s.io/utils v0.0.0-20240310230437-4693a0247e57 // indirect
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect
sigs.k8s.io/yaml v1.3.0 // indirect
sigs.k8s.io/yaml v1.4.0 // indirect
)

229
go.sum
View File

@@ -1,10 +1,9 @@
github.com/1Password/connect-sdk-go v1.5.3 h1:KyjJ+kCKj6BwB2Y8tPM1Ixg5uIS6HsB0uWA8U38p/Uk=
github.com/1Password/connect-sdk-go v1.5.3/go.mod h1:5rSymY4oIYtS4G3t0oMkGAXBeoYiukV3vkqlnEjIDJs=
github.com/1password/onepassword-sdk-go v0.3.1 h1:dz0LrYuIh/HrZ7rxr8NMymikNLBIXhyj4NBmo5Tdamc=
github.com/1password/onepassword-sdk-go v0.3.1/go.mod h1:kssODrGGqHtniqPR91ZPoCMEo79mKulKat7RaD1bunk=
github.com/HdrHistogram/hdrhistogram-go v1.1.2 h1:5IcZpTvzydCQeHzK4Ef/D5rrSqwxob0t8PQPMybUNFM=
github.com/HdrHistogram/hdrhistogram-go v1.1.2/go.mod h1:yDgFjdqOqDEKOvasDdhWNXYg9BVp4O+o5f6V/ehm6Oo=
github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA=
github.com/benbjohnson/clock v1.3.0 h1:ip6w0uFQkncKQ979AypyG0ER7mqUSBdKLOgAle/AT8A=
github.com/benbjohnson/clock v1.3.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44=
@@ -12,43 +11,43 @@ github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XL
github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/emicklei/go-restful/v3 v3.11.0 h1:rAQeMHw1c7zTmncogyy8VvRZwtkmkZ4FxERmMY4rD+g=
github.com/emicklei/go-restful/v3 v3.11.0/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc=
github.com/dylibso/observe-sdk/go v0.0.0-20240819160327-2d926c5d788a h1:UwSIFv5g5lIvbGgtf3tVwC7Ky9rmMFBp0RMs+6f6YqE=
github.com/dylibso/observe-sdk/go v0.0.0-20240819160327-2d926c5d788a/go.mod h1:C8DzXehI4zAbrdlbtOByKX6pfivJTBiV9Jjqv56Yd9Q=
github.com/emicklei/go-restful/v3 v3.12.0 h1:y2DdzBAURM29NFF94q6RaY4vjIH1rtwDapwQtU84iWk=
github.com/emicklei/go-restful/v3 v3.12.0/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc=
github.com/evanphx/json-patch v5.6.0+incompatible h1:jBYDEEiFBPxA0v50tFdvOzQQTCvpL6mnFh5mB2/l16U=
github.com/evanphx/json-patch v5.6.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
github.com/evanphx/json-patch/v5 v5.6.0 h1:b91NhWfaz02IuVxO9faSllyAtNXHMPkC5J8sJCLunww=
github.com/evanphx/json-patch/v5 v5.6.0/go.mod h1:G79N1coSVB93tBe7j6PhzjmR3/2VvlbKOFpnXhI9Bw4=
github.com/fsnotify/fsnotify v1.6.0 h1:n+5WquG0fcWoWp6xPWfHdbskMCQaFnG6PfBrh1Ky4HY=
github.com/fsnotify/fsnotify v1.6.0/go.mod h1:sl3t1tCWJFWoRz9R8WJCbQihKKwmorjAbSClcnxKAGw=
github.com/go-logr/logr v1.2.4/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-logr/logr v1.3.0 h1:2y3SDp0ZXuc6/cjLSZ+Q3ir+QB9T/iG5yYRXqsagWSY=
github.com/go-logr/logr v1.3.0/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/zapr v1.2.4 h1:QHVo+6stLbfJmYGkQ7uGHUCu5hnAFAj6mDe6Ea0SeOo=
github.com/go-logr/zapr v1.2.4/go.mod h1:FyHWQIzQORZ0QVE1BtVHv3cKtNLuXsbNLtpuhNapBOA=
github.com/go-openapi/jsonpointer v0.19.6 h1:eCs3fxoIi3Wh6vtgmLTOjdhSpiqphQ+DaPn38N2ZdrE=
github.com/go-openapi/jsonpointer v0.19.6/go.mod h1:osyAmYz/mB/C3I+WsTTSgw1ONzaLJoLCyoi6/zppojs=
github.com/go-openapi/jsonreference v0.20.2 h1:3sVjiK66+uXK/6oQ8xgcRKcFgQ5KXa2KvnJRumpMGbE=
github.com/go-openapi/jsonreference v0.20.2/go.mod h1:Bl1zwGIM8/wsvqjsOQLJ/SH+En5Ap4rVB5KVcIDZG2k=
github.com/go-openapi/swag v0.22.3 h1:yMBqmnQ0gyZvEb/+KzuWZOXgllrXT4SADYbvDaXHv/g=
github.com/go-openapi/swag v0.22.3/go.mod h1:UzaqsxGiab7freDnrUUra0MwWfN/q7tE4j+VcZ0yl14=
github.com/evanphx/json-patch/v5 v5.9.0 h1:kcBlZQbplgElYIlo/n1hJbls2z/1awpXxpRi0/FOJfg=
github.com/evanphx/json-patch/v5 v5.9.0/go.mod h1:VNkHZ/282BpEyt/tObQO8s5CMPmYYq14uClGH4abBuQ=
github.com/extism/go-sdk v1.7.0 h1:yHbSa2JbcF60kjGsYiGEOcClfbknqCJchyh9TRibFWo=
github.com/extism/go-sdk v1.7.0/go.mod h1:Dhuc1qcD0aqjdqJ3ZDyGdkZPEj/EHKVjbE4P+1XRMqc=
github.com/fsnotify/fsnotify v1.7.0 h1:8JEhPFa5W2WU7YfeZzPNqzMP6Lwt7L2715Ggo0nosvA=
github.com/fsnotify/fsnotify v1.7.0/go.mod h1:40Bi/Hjc2AVfZrqy+aj+yEI+/bRxZnMJyTJwOpGvigM=
github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/zapr v1.3.0 h1:XGdV8XW8zdwFiwOA2Dryh1gj2KRQyOOoNmBy4EplIcQ=
github.com/go-logr/zapr v1.3.0/go.mod h1:YKepepNBd1u/oyhd/yQmtjVXmm9uML4IXUgMOwR8/Gg=
github.com/go-openapi/jsonpointer v0.21.0 h1:YgdVicSA9vH5RiHs9TZW5oyafXZFc6+2Vc1rr/O9oNQ=
github.com/go-openapi/jsonpointer v0.21.0/go.mod h1:IUyH9l/+uyhIYQ/PXVA41Rexl+kOkAPDdXEYns6fzUY=
github.com/go-openapi/jsonreference v0.21.0 h1:Rs+Y7hSXT83Jacb7kFyjn4ijOuVGSvOdF2+tg1TRrwQ=
github.com/go-openapi/jsonreference v0.21.0/go.mod h1:LmZmgsrTkVg9LG4EaHeY8cBDslNPMo06cago5JNLkm4=
github.com/go-openapi/swag v0.23.0 h1:vsEVJDUo2hPJ2tu0/Xc+4noaxyEffXNIs3cOULZ+GrE=
github.com/go-openapi/swag v0.23.0/go.mod h1:esZ8ITTYEsH1V2trKHjAN8Ai7xHb8RV+YSZ577vPjgQ=
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 h1:tfuBGBXKqDEevZMzYi5KSi8KkcZtzBcTgAUUtapy0OI=
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572/go.mod h1:9Pwr4B2jHnOSGXyyzV8ROjYa2ojvAY6HCGYYfMoC3Ls=
github.com/gobwas/glob v0.2.3 h1:A4xDbljILXROh+kObIiy5kIaPYD8e96x1tgBhUI5J+Y=
github.com/gobwas/glob v0.2.3/go.mod h1:d3Ez4x06l9bZtSvzIay5+Yzi0fmZzPgnTbPcKjJAkT8=
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg=
github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/gnostic-models v0.6.8 h1:yo/ABAfM5IMRsS1VnXjTBvUb61tFIHozhlYvRgGre9I=
github.com/google/gnostic-models v0.6.8/go.mod h1:5n7qKqH0f5wFt+aWF8CW6pZLLNOfYuF5OpfBSENuI8U=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
@@ -57,30 +56,25 @@ github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0=
github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1 h1:K6RDEckDVWvDI9JAJYCmNdQXq6neHJOYx3V6jnqNEec=
github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=
github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/imdario/mergo v0.3.6 h1:xTNEAn+kxVO7dTZGu0CegyqKZmoWFI0rF8UxjlB2d28=
github.com/imdario/mergo v0.3.6/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=
github.com/ianlancetaylor/demangle v0.0.0-20240805132620-81f5be970eca h1:T54Ema1DU8ngI+aef9ZhAhNGQhcRTrWxVeG07F+c/Rw=
github.com/ianlancetaylor/demangle v0.0.0-20240805132620-81f5be970eca/go.mod h1:gx7rwoVhcfuVKG5uya9Hs3Sxj7EIvldVofAWIUtGouw=
github.com/imdario/mergo v0.3.16 h1:wwQJbIsHYGMUyLSPrEq1CT16AhnhNJQ51+4fdHUnCl4=
github.com/imdario/mergo v0.3.16/go.mod h1:WBLT9ZmE3lPoWsEzCh9LPo3TiwVN+ZKEjmz+hD27ysY=
github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY=
github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y=
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0=
github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
github.com/matttproud/golang_protobuf_extensions v1.0.4 h1:mmDVorXM7PCGKw94cs5zkfA9PSy5pEvNWRP0ET0TIVo=
github.com/matttproud/golang_protobuf_extensions v1.0.4/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
@@ -88,129 +82,102 @@ github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9G
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/onsi/ginkgo/v2 v2.13.2 h1:Bi2gGVkfn6gQcjNjZJVO8Gf0FHzMPf2phUei9tejVMs=
github.com/onsi/ginkgo/v2 v2.13.2/go.mod h1:XStQ8QcGwLyF4HdfcZB8SFOS/MWCgDuXMSBe6zrvLgM=
github.com/onsi/ginkgo/v2 v2.14.0 h1:vSmGj2Z5YPb9JwCWT6z6ihcUvDhuXLc3sJiqd3jMKAY=
github.com/onsi/ginkgo/v2 v2.14.0/go.mod h1:JkUdW7JkN0V6rFvsHcJ478egV3XH9NxpD27Hal/PhZw=
github.com/onsi/gomega v1.30.0 h1:hvMK7xYz4D3HapigLTeGdId/NcfQx1VHMJc60ew99+8=
github.com/onsi/gomega v1.30.0/go.mod h1:9sxs+SwGrKI0+PWe4Fxa9tFQQBG5xSsSbMXOI8PPpoQ=
github.com/opentracing/opentracing-go v1.2.0 h1:uEJPy/1a5RIPAJ0Ov+OIO8OxWu77jEv+1B0VhjKrZUs=
github.com/opentracing/opentracing-go v1.2.0/go.mod h1:GxEUsuufX4nBwe+T+Wl9TAgYrxe9dPLANfrWvHYVTgc=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v1.16.0 h1:yk/hx9hDbrGHovbci4BY+pRMfSuuat626eFsHb7tmT8=
github.com/prometheus/client_golang v1.16.0/go.mod h1:Zsulrv/L9oM40tJ7T815tM89lFEugiJ9HzIqaAx4LKc=
github.com/prometheus/client_model v0.4.0 h1:5lQXD3cAg1OXBf4Wq03gTrXHeaV0TQvGfUooCfx1yqY=
github.com/prometheus/client_model v0.4.0/go.mod h1:oMQmHW1/JoDwqLtg57MGgP/Fb1CJEYF2imWWhWtMkYU=
github.com/prometheus/common v0.44.0 h1:+5BrQJwiBB9xsMygAB3TNvpQKOwlkc25LbISbrdOOfY=
github.com/prometheus/common v0.44.0/go.mod h1:ofAIvZbQ1e/nugmZGz4/qCb9Ap1VoSTIO7x0VV9VvuY=
github.com/prometheus/procfs v0.10.1 h1:kYK1Va/YMlutzCGazswoHKo//tZVlFpKYh+PymziUAg=
github.com/prometheus/procfs v0.10.1/go.mod h1:nwNm2aOCAYw8uTR/9bWRREkZFxAUcWzPHWJq+XBB/FM=
github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ=
github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog=
github.com/prometheus/client_golang v1.19.0 h1:ygXvpU1AoN1MhdzckN+PyD9QJOSD4x7kmXYlnfbA6JU=
github.com/prometheus/client_golang v1.19.0/go.mod h1:ZRM9uEAypZakd+q/x7+gmsvXdURP+DABIEIjnmDdp+k=
github.com/prometheus/client_model v0.6.0 h1:k1v3CzpSRUTrKMppY35TLwPvxHqBu0bYgxZzqGIgaos=
github.com/prometheus/client_model v0.6.0/go.mod h1:NTQHnmxFpouOD0DpvP4XujX3CdOAGQPoaGhyTchlyt8=
github.com/prometheus/common v0.51.1 h1:eIjN50Bwglz6a/c3hAgSMcofL3nD+nFQkV6Dd4DsQCw=
github.com/prometheus/common v0.51.1/go.mod h1:lrWtQx+iDfn2mbH5GUzlH9TSHyfZpHkSiG1W7y3sF2Q=
github.com/prometheus/procfs v0.13.0 h1:GqzLlQyfsPbaEHaQkO7tbDlriv/4o5Hudv6OXHGKX7o=
github.com/prometheus/procfs v0.13.0/go.mod h1:cd4PFCR54QLnGKPaKGA6l+cfuNXtht43ZKY6tow0Y1g=
github.com/rogpeppe/go-internal v1.11.0 h1:cWPaGQEPrBb5/AsnsZesgZZ9yb1OQ+GOISoDNXVBh4M=
github.com/rogpeppe/go-internal v1.11.0/go.mod h1:ddIwULY96R17DhadqLgMfk9H9tvdUzkipdSkR5nkCZA=
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0 h1:1zr/of2m5FGMsad5YfcqgdqdWrIhu+EBEJRhR1U7z/c=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/tetratelabs/wabin v0.0.0-20230304001439-f6f874872834 h1:ZF+QBjOI+tILZjBaFj3HgFonKXUcwgJ4djLb6i42S3Q=
github.com/tetratelabs/wabin v0.0.0-20230304001439-f6f874872834/go.mod h1:m9ymHTgNSEjuxvw8E7WWe4Pl4hZQHXONY8wE6dMLaRk=
github.com/tetratelabs/wazero v1.9.0 h1:IcZ56OuxrtaEz8UYNRHBrUa9bYeX9oVY93KspZZBf/I=
github.com/tetratelabs/wazero v1.9.0/go.mod h1:TSbcXCfFP0L2FGkRPxHphadXPjo1T6W+CseNNY7EkjM=
github.com/uber/jaeger-client-go v2.30.0+incompatible h1:D6wyKGCecFaSRUpo8lCVbaOOb6ThwMmTEbhRwtKR97o=
github.com/uber/jaeger-client-go v2.30.0+incompatible/go.mod h1:WVhlPFC8FDjOFMMWRy2pZqQJSXxYSwNYOkTr/Z6d3Kk=
github.com/uber/jaeger-lib v2.4.1+incompatible h1:td4jdvLcExb4cBISKIpHuGoVXh+dVKhn2Um6rjCsSsg=
github.com/uber/jaeger-lib v2.4.1+incompatible/go.mod h1:ComeNDZlWwrWnDv8aPp0Ba6+uUTzImX/AauajbLI56U=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
go.uber.org/atomic v1.10.0 h1:9qC72Qh0+3MqyJbAn8YU5xVq1frD8bn3JtD2oXtafVQ=
go.uber.org/atomic v1.10.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=
go.uber.org/goleak v1.1.11/go.mod h1:cwTWslyiVhfpKIDGSZEM2HlOvcqm+tG4zioyIeLoqMQ=
go.uber.org/goleak v1.2.1 h1:NBol2c7O1ZokfZ0LEU9K6Whx/KnwvepVetCUhtKja4A=
go.uber.org/goleak v1.2.1/go.mod h1:qlT2yGI9QafXHhZZLxlSuNsMw3FFLxBr+tBRlmO1xH4=
go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU=
go.opentelemetry.io/proto/otlp v1.3.1 h1:TrMUixzpM0yuc/znrFTP9MMRh8trP93mkCiDVeXrui0=
go.opentelemetry.io/proto/otlp v1.3.1/go.mod h1:0X1WI4de4ZsLrrJNLAQbFeLCm3T7yBkR0XqQ7niQU+8=
go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE=
go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/zap v1.24.0/go.mod h1:2kMP+WWQ8aoFoedH3T2sq6iJ2yDWpHbP0f6MQbS9Gkg=
go.uber.org/zap v1.25.0 h1:4Hvk6GtkucQ790dqmj7l1eEnRdKm3k3ZUrUMS2d5+5c=
go.uber.org/zap v1.25.0/go.mod h1:JIAUzQIH94IC4fOJQm7gMmBJP5k7wQfdcnYdPoEXJYk=
go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8=
go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/exp v0.0.0-20220722155223-a9213eeb770e h1:+WEEuIdZHnUeJJmEUjyYC2gfUMj69yZXw17EnHg/otA=
golang.org/x/exp v0.0.0-20220722155223-a9213eeb770e/go.mod h1:Kr81I6Kryrl9sr8s2FK3vxD90NdsKWRuOIl2O4CvYbA=
golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/exp v0.0.0-20240318143956-a85f2c67cd81 h1:6R2FC06FonbXQ8pK11/PDFY6N6LWlf9KlzibaCapmqc=
golang.org/x/exp v0.0.0-20240318143956-a85f2c67cd81/go.mod h1:CQ1k9gNrJ50XIzaKCRR2hssIjF07kZFEiieALBM/ARQ=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.13.0 h1:I/DsJXRlw/8l/0c24sM9yb0T4z9liZTduXvdAWYiysY=
golang.org/x/mod v0.13.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM=
golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM=
golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE=
golang.org/x/oauth2 v0.10.0 h1:zHCpF2Khkwy4mMB4bv0U37YtJdTGW8jI0glAApi0Kh8=
golang.org/x/oauth2 v0.10.0/go.mod h1:kTpgurOux7LqtuxjuyZa4Gj2gdezIt/jQtGnNFfypQI=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/net v0.41.0 h1:vBTly1HeNPEn3wtREYfy4GZ/NECgw2Cnl+nK6Nz3uvw=
golang.org/x/net v0.41.0/go.mod h1:B/K4NNqkfmg07DQYrbwvSluqCJOOXwUjeb/5lOisjbA=
golang.org/x/oauth2 v0.30.0 h1:dnDm7JmhM45NNpd8FDDeLhK6FwqbOf4MLCM9zb1BOHI=
golang.org/x/oauth2 v0.30.0/go.mod h1:B++QgG3ZKulg6sRPGD/mqlHQs5rB3Ml9erfeDY7xKlU=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220908164124-27713097b956/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.14.0 h1:Vz7Qs629MkJkGyHxUlRHizWJRG2j8fbQKjELVSNhy7Q=
golang.org/x/sys v0.14.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.13.0 h1:bb+I9cTfFazGW51MZqBVmZy7+JEJMouUHTUSKVQLBek=
golang.org/x/term v0.13.0/go.mod h1:LTmsnFJwVN6bCy1rVCoS+qHT1HhALEFxKncY3WNNh4U=
golang.org/x/sys v0.33.0 h1:q3i8TbbEz+JRD9ywIRlyRAQbM0qF7hu24q3teo2hbuw=
golang.org/x/sys v0.33.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/term v0.32.0 h1:DR4lr0TjUs3epypdhTOkMmuF5CDFJ/8pOnbzMZPQ7bg=
golang.org/x/term v0.32.0/go.mod h1:uZG1FhGx848Sqfsq4/DlJr3xGGsYMu/L5GW4abiaEPQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.13.0 h1:ablQoSUd0tRdKxZewP80B+BaqeKJuVhuRxj/dkrun3k=
golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
golang.org/x/time v0.3.0 h1:rg5rLMjNzMS1RkNLzCG38eapWhnYLFYXDXj2gOlr8j4=
golang.org/x/time v0.3.0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/text v0.26.0 h1:P42AVeLghgTYr4+xUnTRKDMqpar+PtX7KWuNQL21L8M=
golang.org/x/text v0.26.0/go.mod h1:QK15LZJUUQVJxhz7wXgxSy/CJaTFjd0G+YLonydOVQA=
golang.org/x/time v0.5.0 h1:o7cqy6amK/52YcAKIPlM3a+Fpj35zvRj2TP+e1xFSfk=
golang.org/x/time v0.5.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.14.0 h1:jvNa2pY0M4r62jkRQ6RwEZZyPcymeL9XZMLBbV7U2nc=
golang.org/x/tools v0.14.0/go.mod h1:uYBEerGOWcJyEORxN+Ek8+TT266gXkNlHdJBwexUsBg=
golang.org/x/tools v0.33.0 h1:4qz2S3zmRxbGIhDIAgjxvFutSvH5EfnsYrRBj0UI0bc=
golang.org/x/tools v0.33.0/go.mod h1:CIJMaWEY88juyUfo7UbgPqbC8rU2OqfAV1h2Qp0oMYI=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gomodules.xyz/jsonpatch/v2 v2.4.0 h1:Ci3iUJyx9UeRx7CeFN8ARgGbkESwJK+KB9lLcWxY/Zw=
gomodules.xyz/jsonpatch/v2 v2.4.0/go.mod h1:AH3dM2RI6uoBZxn3LVrfvJ3E0/9dG4cSrbuBJT4moAY=
google.golang.org/appengine v1.6.7 h1:FZR1q0exgwxzPzp/aF+VccGrSfxfPpkBqjIIEq3ru6c=
google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8=
google.golang.org/protobuf v1.31.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg=
google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc=
@@ -221,29 +188,29 @@ gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
k8s.io/api v0.29.0 h1:NiCdQMY1QOp1H8lfRyeEf8eOwV6+0xA6XEE44ohDX2A=
k8s.io/api v0.29.0/go.mod h1:sdVmXoz2Bo/cb77Pxi71IPTSErEW32xa4aXwKH7gfBA=
k8s.io/apiextensions-apiserver v0.28.3 h1:Od7DEnhXHnHPZG+W9I97/fSQkVpVPQx2diy+2EtmY08=
k8s.io/apiextensions-apiserver v0.28.3/go.mod h1:NE1XJZ4On0hS11aWWJUTNkmVB03j9LM7gJSisbRt8Lc=
k8s.io/apimachinery v0.29.0 h1:+ACVktwyicPz0oc6MTMLwa2Pw3ouLAfAon1wPLtG48o=
k8s.io/apimachinery v0.29.0/go.mod h1:eVBxQ/cwiJxH58eK/jd/vAk4mrxmVlnpBH5J2GbMeis=
k8s.io/client-go v0.29.0 h1:KmlDtFcrdUzOYrBhXHgKw5ycWzc3ryPX5mQe0SkG3y8=
k8s.io/client-go v0.29.0/go.mod h1:yLkXH4HKMAywcrD82KMSmfYg2DlE8mepPR4JGSo5n38=
k8s.io/component-base v0.29.0 h1:T7rjd5wvLnPBV1vC4zWd/iWRbV8Mdxs+nGaoaFzGw3s=
k8s.io/component-base v0.29.0/go.mod h1:sADonFTQ9Zc9yFLghpDpmNXEdHyQmFIGbiuZbqAXQ1M=
k8s.io/klog/v2 v2.110.1 h1:U/Af64HJf7FcwMcXyKm2RPM22WZzyR7OSpYj5tg3cL0=
k8s.io/klog/v2 v2.110.1/go.mod h1:YGtd1984u+GgbuZ7e08/yBuAfKLSO0+uR1Fhi6ExXjo=
k8s.io/kube-openapi v0.0.0-20231010175941-2dd684a91f00 h1:aVUu9fTY98ivBPKR9Y5w/AuzbMm96cd3YHRTU83I780=
k8s.io/kube-openapi v0.0.0-20231010175941-2dd684a91f00/go.mod h1:AsvuZPBlUDVuCdzJ87iajxtXuR9oktsTctW/R9wwouA=
k8s.io/api v0.29.3 h1:2ORfZ7+bGC3YJqGpV0KSDDEVf8hdGQ6A03/50vj8pmw=
k8s.io/api v0.29.3/go.mod h1:y2yg2NTyHUUkIoTC+phinTnEa3KFM6RZ3szxt014a80=
k8s.io/apiextensions-apiserver v0.29.3 h1:9HF+EtZaVpFjStakF4yVufnXGPRppWFEQ87qnO91YeI=
k8s.io/apiextensions-apiserver v0.29.3/go.mod h1:po0XiY5scnpJfFizNGo6puNU6Fq6D70UJY2Cb2KwAVc=
k8s.io/apimachinery v0.29.3 h1:2tbx+5L7RNvqJjn7RIuIKu9XTsIZ9Z5wX2G22XAa5EU=
k8s.io/apimachinery v0.29.3/go.mod h1:hx/S4V2PNW4OMg3WizRrHutyB5la0iCUbZym+W0EQIU=
k8s.io/client-go v0.29.3 h1:R/zaZbEAxqComZ9FHeQwOh3Y1ZUs7FaHKZdQtIc2WZg=
k8s.io/client-go v0.29.3/go.mod h1:tkDisCvgPfiRpxGnOORfkljmS+UrW+WtXAy2fTvXJB0=
k8s.io/component-base v0.29.3 h1:Oq9/nddUxlnrCuuR2K/jp6aflVvc0uDvxMzAWxnGzAo=
k8s.io/component-base v0.29.3/go.mod h1:Yuj33XXjuOk2BAaHsIGHhCKZQAgYKhqIxIjIr2UXYio=
k8s.io/klog/v2 v2.120.1 h1:QXU6cPEOIslTGvZaXvFWiP9VKyeet3sawzTOvdXb4Vw=
k8s.io/klog/v2 v2.120.1/go.mod h1:3Jpz1GvMt720eyJH1ckRHK1EDfpxISzJ7I9OYgaDtPE=
k8s.io/kube-openapi v0.0.0-20240322212309-b815d8309940 h1:qVoMaQV5t62UUvHe16Q3eb2c5HPzLHYzsi0Tu/xLndo=
k8s.io/kube-openapi v0.0.0-20240322212309-b815d8309940/go.mod h1:yD4MZYeKMBwQKVht279WycxKyM84kkAx2DPrTXaeb98=
k8s.io/kubectl v0.29.0 h1:Oqi48gXjikDhrBF67AYuZRTcJV4lg2l42GmvsP7FmYI=
k8s.io/kubectl v0.29.0/go.mod h1:0jMjGWIcMIQzmUaMgAzhSELv5WtHo2a8pq67DtviAJs=
k8s.io/utils v0.0.0-20230726121419-3b25d923346b h1:sgn3ZU783SCgtaSJjpcVVlRqd6GSnlTLKgpAAttJvpI=
k8s.io/utils v0.0.0-20230726121419-3b25d923346b/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0=
sigs.k8s.io/controller-runtime v0.16.3 h1:2TuvuokmfXvDUamSx1SuAOO3eTyye+47mJCigwG62c4=
sigs.k8s.io/controller-runtime v0.16.3/go.mod h1:j7bialYoSn142nv9sCOJmQgDXQXxnroFU4VnX/brVJ0=
k8s.io/utils v0.0.0-20240310230437-4693a0247e57 h1:gbqbevonBh57eILzModw6mrkbwM0gQBEuevE/AaBsHY=
k8s.io/utils v0.0.0-20240310230437-4693a0247e57/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0=
sigs.k8s.io/controller-runtime v0.17.2 h1:FwHwD1CTUemg0pW2otk7/U5/i5m2ymzvOXdbeGOUvw0=
sigs.k8s.io/controller-runtime v0.17.2/go.mod h1:+MngTvIQQQhfXtwfdGw/UOQ/aIaqsYywfCINOtwMO/s=
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd h1:EDPBXCAspyGV4jQlpZSudPeMmr1bNJefnuqLsRAsHZo=
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd/go.mod h1:B8JuhiUyNFVKdsE8h686QcCxMaH6HrOAZj4vswFpcB0=
sigs.k8s.io/structured-merge-diff/v4 v4.4.1 h1:150L+0vs/8DA78h1u02ooW1/fFq/Lwr+sGiqlzvrtq4=
sigs.k8s.io/structured-merge-diff/v4 v4.4.1/go.mod h1:N8hJocpFajUSSeSJ9bOZ77VzejKZaXsTtZo4/u7Io08=
sigs.k8s.io/yaml v1.3.0 h1:a2VclLzOGrwOHDiV8EfBGhvjHvP46CtW5j6POvhYGGo=
sigs.k8s.io/yaml v1.3.0/go.mod h1:GeOyir5tyXNByN85N/dRIT9es5UQNerPYEKK56eTBm8=
sigs.k8s.io/yaml v1.4.0 h1:Mk1wCc2gy/F0THH0TAp1QYyJNzRm2KCLy3o5ASXVI5E=
sigs.k8s.io/yaml v1.4.0/go.mod h1:Ejl7/uTz7PSA4eKMyQCUTnhZYNmLIl+5c2lQPGR2BPY=

View File

@@ -28,14 +28,13 @@ import (
"context"
"fmt"
"regexp"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
"github.com/1Password/connect-sdk-go/connect"
"strings"
"time"
kubeSecrets "github.com/1Password/onepassword-operator/pkg/kubernetessecrets"
"github.com/1Password/onepassword-operator/pkg/logs"
op "github.com/1Password/onepassword-operator/pkg/onepassword"
opclient "github.com/1Password/onepassword-operator/pkg/onepassword/client"
"github.com/1Password/onepassword-operator/pkg/utils"
appsv1 "k8s.io/api/apps/v1"
@@ -47,6 +46,7 @@ import (
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/client/apiutil"
logf "sigs.k8s.io/controller-runtime/pkg/log"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
)
var logDeployment = logf.Log.WithName("controller_deployment")
@@ -55,7 +55,7 @@ var logDeployment = logf.Log.WithName("controller_deployment")
type DeploymentReconciler struct {
client.Client
Scheme *runtime.Scheme
OpConnectClient connect.Client
OpClient opclient.Client
OpAnnotationRegExp *regexp.Regexp
}
@@ -77,7 +77,7 @@ func (r *DeploymentReconciler) Reconcile(ctx context.Context, req ctrl.Request)
reqLogger.V(logs.DebugLevel).Info("Reconciling Deployment")
deployment := &appsv1.Deployment{}
err := r.Get(context.Background(), req.NamespacedName, deployment)
err := r.Get(ctx, req.NamespacedName, deployment)
if err != nil {
if errors.IsNotFound(err) {
return reconcile.Result{}, nil
@@ -97,13 +97,18 @@ func (r *DeploymentReconciler) Reconcile(ctx context.Context, req ctrl.Request)
// This is so we can handle cleanup of associated secrets properly
if !utils.ContainsString(deployment.ObjectMeta.Finalizers, finalizer) {
deployment.ObjectMeta.Finalizers = append(deployment.ObjectMeta.Finalizers, finalizer)
if err = r.Update(context.Background(), deployment); err != nil {
if err = r.Update(ctx, deployment); err != nil {
return reconcile.Result{}, err
}
}
// Handles creation or updating secrets for deployment if needed
if err = r.handleApplyingDeployment(deployment, deployment.Namespace, annotations, req); err != nil {
return ctrl.Result{}, err
if err = r.handleApplyingDeployment(ctx, deployment, deployment.Namespace, annotations, req); err != nil {
if strings.Contains(err.Error(), "rate limit") {
reqLogger.V(logs.InfoLevel).Info("1Password rate limit hit. Requeuing after 15 minutes.")
return ctrl.Result{RequeueAfter: 15 * time.Minute}, nil
} else {
return ctrl.Result{}, err
}
}
return ctrl.Result{}, nil
}
@@ -112,12 +117,12 @@ func (r *DeploymentReconciler) Reconcile(ctx context.Context, req ctrl.Request)
if utils.ContainsString(deployment.ObjectMeta.Finalizers, finalizer) {
secretName := annotations[op.NameAnnotation]
if err = r.cleanupKubernetesSecretForDeployment(secretName, deployment); err != nil {
if err = r.cleanupKubernetesSecretForDeployment(ctx, secretName, deployment); err != nil {
return ctrl.Result{}, err
}
// Remove the finalizer from the deployment so deletion of deployment can be completed
if err = r.removeOnePasswordFinalizerFromDeployment(deployment); err != nil {
if err = r.removeOnePasswordFinalizerFromDeployment(ctx, deployment); err != nil {
return reconcile.Result{}, err
}
}
@@ -131,7 +136,7 @@ func (r *DeploymentReconciler) SetupWithManager(mgr ctrl.Manager) error {
Complete(r)
}
func (r *DeploymentReconciler) cleanupKubernetesSecretForDeployment(secretName string, deletedDeployment *appsv1.Deployment) error {
func (r *DeploymentReconciler) cleanupKubernetesSecretForDeployment(ctx context.Context, secretName string, deletedDeployment *appsv1.Deployment) error {
kubernetesSecret := &corev1.Secret{}
kubernetesSecret.ObjectMeta.Name = secretName
kubernetesSecret.ObjectMeta.Namespace = deletedDeployment.Namespace
@@ -141,14 +146,14 @@ func (r *DeploymentReconciler) cleanupKubernetesSecretForDeployment(secretName s
}
updatedSecrets := map[string]*corev1.Secret{secretName: kubernetesSecret}
multipleDeploymentsUsingSecret, err := r.areMultipleDeploymentsUsingSecret(updatedSecrets, *deletedDeployment)
multipleDeploymentsUsingSecret, err := r.areMultipleDeploymentsUsingSecret(ctx, updatedSecrets, *deletedDeployment)
if err != nil {
return err
}
// Only delete the associated kubernetes secret if it is not being used by other deployments
if !multipleDeploymentsUsingSecret {
if err = r.Delete(context.Background(), kubernetesSecret); err != nil {
if err = r.Delete(ctx, kubernetesSecret); err != nil {
if !errors.IsNotFound(err) {
return err
}
@@ -157,13 +162,13 @@ func (r *DeploymentReconciler) cleanupKubernetesSecretForDeployment(secretName s
return nil
}
func (r *DeploymentReconciler) areMultipleDeploymentsUsingSecret(updatedSecrets map[string]*corev1.Secret, deletedDeployment appsv1.Deployment) (bool, error) {
func (r *DeploymentReconciler) areMultipleDeploymentsUsingSecret(ctx context.Context, updatedSecrets map[string]*corev1.Secret, deletedDeployment appsv1.Deployment) (bool, error) {
deployments := &appsv1.DeploymentList{}
opts := []client.ListOption{
client.InNamespace(deletedDeployment.Namespace),
}
err := r.List(context.Background(), deployments, opts...)
err := r.List(ctx, deployments, opts...)
if err != nil {
logDeployment.Error(err, "Failed to list kubernetes deployments")
return false, err
@@ -179,12 +184,12 @@ func (r *DeploymentReconciler) areMultipleDeploymentsUsingSecret(updatedSecrets
return false, nil
}
func (r *DeploymentReconciler) removeOnePasswordFinalizerFromDeployment(deployment *appsv1.Deployment) error {
func (r *DeploymentReconciler) removeOnePasswordFinalizerFromDeployment(ctx context.Context, deployment *appsv1.Deployment) error {
deployment.ObjectMeta.Finalizers = utils.RemoveString(deployment.ObjectMeta.Finalizers, finalizer)
return r.Update(context.Background(), deployment)
return r.Update(ctx, deployment)
}
func (r *DeploymentReconciler) handleApplyingDeployment(deployment *appsv1.Deployment, namespace string, annotations map[string]string, request reconcile.Request) error {
func (r *DeploymentReconciler) handleApplyingDeployment(ctx context.Context, deployment *appsv1.Deployment, namespace string, annotations map[string]string, request reconcile.Request) error {
reqLog := logDeployment.WithValues("Request.Namespace", request.Namespace, "Request.Name", request.Name)
secretName := annotations[op.NameAnnotation]
@@ -196,15 +201,15 @@ func (r *DeploymentReconciler) handleApplyingDeployment(deployment *appsv1.Deplo
return nil
}
item, err := op.GetOnePasswordItemByPath(r.OpConnectClient, annotations[op.ItemPathAnnotation])
item, err := op.GetOnePasswordItemByPath(ctx, r.OpClient, annotations[op.ItemPathAnnotation])
if err != nil {
return fmt.Errorf("Failed to retrieve item: %v", err)
return fmt.Errorf("failed to retrieve item: %w", err)
}
// Create owner reference.
gvk, err := apiutil.GVKForObject(deployment, r.Scheme)
if err != nil {
return fmt.Errorf("could not to retrieve group version kind: %v", err)
return fmt.Errorf("could not to retrieve group version kind: %w", err)
}
ownerRef := &metav1.OwnerReference{
APIVersion: gvk.GroupVersion().String(),
@@ -213,5 +218,5 @@ func (r *DeploymentReconciler) handleApplyingDeployment(deployment *appsv1.Deplo
UID: deployment.GetUID(),
}
return kubeSecrets.CreateKubernetesSecretFromItem(r.Client, secretName, namespace, item, annotations[op.RestartDeploymentsAnnotation], secretLabels, secretType, ownerRef)
return kubeSecrets.CreateKubernetesSecretFromItem(ctx, r.Client, secretName, namespace, item, annotations[op.RestartDeploymentsAnnotation], secretLabels, secretType, ownerRef)
}

View File

@@ -2,9 +2,6 @@ package controller
import (
"context"
"github.com/1Password/connect-sdk-go/onepassword"
"github.com/1Password/onepassword-operator/pkg/mocks"
op "github.com/1Password/onepassword-operator/pkg/onepassword"
"time"
. "github.com/onsi/ginkgo/v2"
@@ -17,6 +14,7 @@ import (
"sigs.k8s.io/controller-runtime/pkg/client"
onepasswordv1 "github.com/1Password/onepassword-operator/api/v1"
op "github.com/1Password/onepassword-operator/pkg/onepassword"
)
const (
@@ -26,14 +24,13 @@ const (
)
var _ = Describe("Deployment controller", func() {
var ctx context.Context
ctx := context.Background()
var deploymentKey types.NamespacedName
var secretKey types.NamespacedName
var deploymentResource *appsv1.Deployment
createdSecret := &v1.Secret{}
makeDeployment := func() {
ctx = context.Background()
deploymentKey = types.NamespacedName{
Name: deploymentName,
@@ -95,28 +92,19 @@ var _ = Describe("Deployment controller", func() {
cleanK8sResources := func() {
// failed test runs that don't clean up leave resources behind.
err := k8sClient.DeleteAllOf(context.Background(), &onepasswordv1.OnePasswordItem{}, client.InNamespace(namespace))
err := k8sClient.DeleteAllOf(ctx, &onepasswordv1.OnePasswordItem{}, client.InNamespace(namespace))
Expect(err).ToNot(HaveOccurred())
err = k8sClient.DeleteAllOf(context.Background(), &v1.Secret{}, client.InNamespace(namespace))
err = k8sClient.DeleteAllOf(ctx, &v1.Secret{}, client.InNamespace(namespace))
Expect(err).ToNot(HaveOccurred())
err = k8sClient.DeleteAllOf(context.Background(), &appsv1.Deployment{}, client.InNamespace(namespace))
err = k8sClient.DeleteAllOf(ctx, &appsv1.Deployment{}, client.InNamespace(namespace))
Expect(err).ToNot(HaveOccurred())
}
mockGetItemFunc := func() {
mocks.DoGetItemFunc = func(uuid string, vaultUUID string) (*onepassword.Item, error) {
item := onepassword.Item{}
item.Fields = []*onepassword.ItemField{}
for k, v := range item1.Data {
item.Fields = append(item.Fields, &onepassword.ItemField{Label: k, Value: v})
}
item.Version = item1.Version
item.Vault.ID = vaultUUID
item.ID = uuid
return &item, nil
}
// mock GetItemByID to return test item 'item1'
mockGetItemByIDFunc.Return(item1.ToModel(), nil)
}
BeforeEach(func() {
@@ -151,17 +139,10 @@ var _ = Describe("Deployment controller", func() {
It("Should update existing K8s Secret using deployment", func() {
By("Updating secret")
mocks.DoGetItemFunc = func(uuid string, vaultUUID string) (*onepassword.Item, error) {
item := onepassword.Item{}
item.Fields = []*onepassword.ItemField{}
for k, v := range item2.Data {
item.Fields = append(item.Fields, &onepassword.ItemField{Label: k, Value: v})
}
item.Version = item2.Version
item.Vault.ID = vaultUUID
item.ID = uuid
return &item, nil
}
// mock GetItemByID to return test item 'item2'
mockGetItemByIDFunc.Return(item2.ToModel(), nil)
Eventually(func() error {
updatedDeployment := &appsv1.Deployment{
TypeMeta: metav1.TypeMeta{

View File

@@ -27,13 +27,14 @@ package controller
import (
"context"
"fmt"
"github.com/1Password/connect-sdk-go/connect"
"strings"
"time"
onepasswordv1 "github.com/1Password/onepassword-operator/api/v1"
kubeSecrets "github.com/1Password/onepassword-operator/pkg/kubernetessecrets"
"github.com/1Password/onepassword-operator/pkg/logs"
op "github.com/1Password/onepassword-operator/pkg/onepassword"
opclient "github.com/1Password/onepassword-operator/pkg/onepassword/client"
"github.com/1Password/onepassword-operator/pkg/utils"
corev1 "k8s.io/api/core/v1"
@@ -52,8 +53,8 @@ var finalizer = "onepassword.com/finalizer.secret"
// OnePasswordItemReconciler reconciles a OnePasswordItem object
type OnePasswordItemReconciler struct {
client.Client
Scheme *runtime.Scheme
OpConnectClient connect.Client
Scheme *runtime.Scheme
OpClient opclient.Client
}
//+kubebuilder:rbac:groups=onepassword.com,resources=onepassworditems,verbs=get;list;watch;create;update;patch;delete
@@ -83,7 +84,7 @@ func (r *OnePasswordItemReconciler) Reconcile(ctx context.Context, req ctrl.Requ
reqLogger.V(logs.DebugLevel).Info("Reconciling OnePasswordItem")
onepassworditem := &onepasswordv1.OnePasswordItem{}
err := r.Get(context.Background(), req.NamespacedName, onepassworditem)
err := r.Get(ctx, req.NamespacedName, onepassworditem)
if err != nil {
if errors.IsNotFound(err) {
return ctrl.Result{}, nil
@@ -97,14 +98,20 @@ func (r *OnePasswordItemReconciler) Reconcile(ctx context.Context, req ctrl.Requ
// This is so we can handle cleanup of associated secrets properly
if !utils.ContainsString(onepassworditem.ObjectMeta.Finalizers, finalizer) {
onepassworditem.ObjectMeta.Finalizers = append(onepassworditem.ObjectMeta.Finalizers, finalizer)
if err = r.Update(context.Background(), onepassworditem); err != nil {
if err = r.Update(ctx, onepassworditem); err != nil {
return ctrl.Result{}, err
}
}
// Handles creation or updating secrets for deployment if needed
err = r.handleOnePasswordItem(onepassworditem, req)
if updateStatusErr := r.updateStatus(onepassworditem, err); updateStatusErr != nil {
err = r.handleOnePasswordItem(ctx, onepassworditem, req)
if err != nil {
if strings.Contains(err.Error(), "rate limit") {
reqLogger.V(logs.InfoLevel).Info("1Password rate limit hit. Requeuing after 15 minutes.")
return ctrl.Result{RequeueAfter: 15 * time.Minute}, nil
}
}
if updateStatusErr := r.updateStatus(ctx, onepassworditem, err); updateStatusErr != nil {
return ctrl.Result{}, fmt.Errorf("cannot update status: %s", updateStatusErr)
}
return ctrl.Result{}, err
@@ -113,12 +120,12 @@ func (r *OnePasswordItemReconciler) Reconcile(ctx context.Context, req ctrl.Requ
if utils.ContainsString(onepassworditem.ObjectMeta.Finalizers, finalizer) {
// Delete associated kubernetes secret
if err = r.cleanupKubernetesSecret(onepassworditem); err != nil {
if err = r.cleanupKubernetesSecret(ctx, onepassworditem); err != nil {
return ctrl.Result{}, err
}
// Remove finalizer now that cleanup is complete
if err = r.removeFinalizer(onepassworditem); err != nil {
if err = r.removeFinalizer(ctx, onepassworditem); err != nil {
return ctrl.Result{}, err
}
}
@@ -132,20 +139,20 @@ func (r *OnePasswordItemReconciler) SetupWithManager(mgr ctrl.Manager) error {
Complete(r)
}
func (r *OnePasswordItemReconciler) removeFinalizer(onePasswordItem *onepasswordv1.OnePasswordItem) error {
func (r *OnePasswordItemReconciler) removeFinalizer(ctx context.Context, onePasswordItem *onepasswordv1.OnePasswordItem) error {
onePasswordItem.ObjectMeta.Finalizers = utils.RemoveString(onePasswordItem.ObjectMeta.Finalizers, finalizer)
if err := r.Update(context.Background(), onePasswordItem); err != nil {
if err := r.Update(ctx, onePasswordItem); err != nil {
return err
}
return nil
}
func (r *OnePasswordItemReconciler) cleanupKubernetesSecret(onePasswordItem *onepasswordv1.OnePasswordItem) error {
func (r *OnePasswordItemReconciler) cleanupKubernetesSecret(ctx context.Context, onePasswordItem *onepasswordv1.OnePasswordItem) error {
kubernetesSecret := &corev1.Secret{}
kubernetesSecret.ObjectMeta.Name = onePasswordItem.Name
kubernetesSecret.ObjectMeta.Namespace = onePasswordItem.Namespace
if err := r.Delete(context.Background(), kubernetesSecret); err != nil {
if err := r.Delete(ctx, kubernetesSecret); err != nil {
if !errors.IsNotFound(err) {
return err
}
@@ -153,26 +160,26 @@ func (r *OnePasswordItemReconciler) cleanupKubernetesSecret(onePasswordItem *one
return nil
}
func (r *OnePasswordItemReconciler) removeOnePasswordFinalizerFromOnePasswordItem(opSecret *onepasswordv1.OnePasswordItem) error {
func (r *OnePasswordItemReconciler) removeOnePasswordFinalizerFromOnePasswordItem(ctx context.Context, opSecret *onepasswordv1.OnePasswordItem) error {
opSecret.ObjectMeta.Finalizers = utils.RemoveString(opSecret.ObjectMeta.Finalizers, finalizer)
return r.Update(context.Background(), opSecret)
return r.Update(ctx, opSecret)
}
func (r *OnePasswordItemReconciler) handleOnePasswordItem(resource *onepasswordv1.OnePasswordItem, req ctrl.Request) error {
func (r *OnePasswordItemReconciler) handleOnePasswordItem(ctx context.Context, resource *onepasswordv1.OnePasswordItem, req ctrl.Request) error {
secretName := resource.GetName()
labels := resource.Labels
secretType := resource.Type
autoRestart := resource.Annotations[op.RestartDeploymentsAnnotation]
item, err := op.GetOnePasswordItemByPath(r.OpConnectClient, resource.Spec.ItemPath)
item, err := op.GetOnePasswordItemByPath(ctx, r.OpClient, resource.Spec.ItemPath)
if err != nil {
return fmt.Errorf("Failed to retrieve item: %v", err)
return fmt.Errorf("failed to retrieve item: %w", err)
}
// Create owner reference.
gvk, err := apiutil.GVKForObject(resource, r.Scheme)
if err != nil {
return fmt.Errorf("could not to retrieve group version kind: %v", err)
return fmt.Errorf("could not to retrieve group version kind: %w", err)
}
ownerRef := &metav1.OwnerReference{
APIVersion: gvk.GroupVersion().String(),
@@ -181,10 +188,10 @@ func (r *OnePasswordItemReconciler) handleOnePasswordItem(resource *onepasswordv
UID: resource.GetUID(),
}
return kubeSecrets.CreateKubernetesSecretFromItem(r.Client, secretName, resource.Namespace, item, autoRestart, labels, secretType, ownerRef)
return kubeSecrets.CreateKubernetesSecretFromItem(ctx, r.Client, secretName, resource.Namespace, item, autoRestart, labels, secretType, ownerRef)
}
func (r *OnePasswordItemReconciler) updateStatus(resource *onepasswordv1.OnePasswordItem, err error) error {
func (r *OnePasswordItemReconciler) updateStatus(ctx context.Context, resource *onepasswordv1.OnePasswordItem, err error) error {
existingCondition := findCondition(resource.Status.Conditions, onepasswordv1.OnePasswordItemReady)
updatedCondition := existingCondition
if err != nil {
@@ -200,7 +207,7 @@ func (r *OnePasswordItemReconciler) updateStatus(resource *onepasswordv1.OnePass
}
resource.Status.Conditions = []onepasswordv1.OnePasswordItemCondition{updatedCondition}
return r.Status().Update(context.Background(), resource)
return r.Status().Update(ctx, resource)
}
func findCondition(conditions []onepasswordv1.OnePasswordItemCondition, t onepasswordv1.OnePasswordItemConditionType) onepasswordv1.OnePasswordItemCondition {

View File

@@ -2,10 +2,7 @@ package controller
import (
"context"
"github.com/1Password/connect-sdk-go/onepassword"
"github.com/1Password/onepassword-operator/pkg/mocks"
"fmt"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
@@ -16,6 +13,7 @@ import (
"sigs.k8s.io/controller-runtime/pkg/reconcile"
onepasswordv1 "github.com/1Password/onepassword-operator/api/v1"
"github.com/1Password/onepassword-operator/pkg/onepassword/model"
)
const (
@@ -32,17 +30,8 @@ var _ = Describe("OnePasswordItem controller", func() {
err = k8sClient.DeleteAllOf(context.Background(), &v1.Secret{}, client.InNamespace(namespace))
Expect(err).ToNot(HaveOccurred())
mocks.DoGetItemFunc = func(uuid string, vaultUUID string) (*onepassword.Item, error) {
item := onepassword.Item{}
item.Fields = []*onepassword.ItemField{}
for k, v := range item1.Data {
item.Fields = append(item.Fields, &onepassword.ItemField{Label: k, Value: v})
}
item.Version = item1.Version
item.Vault.ID = vaultUUID
item.ID = uuid
return &item, nil
}
item := item1.ToModel()
mockGetItemByIDFunc.Return(item, nil)
})
Context("Happy path", func() {
@@ -99,17 +88,13 @@ var _ = Describe("OnePasswordItem controller", func() {
"password": []byte("##newPassword##"),
"extraField": []byte("dev"),
}
mocks.DoGetItemFunc = func(uuid string, vaultUUID string) (*onepassword.Item, error) {
item := onepassword.Item{}
item.Fields = []*onepassword.ItemField{}
for k, v := range newData {
item.Fields = append(item.Fields, &onepassword.ItemField{Label: k, Value: v})
}
item.Version = item1.Version + 1
item.Vault.ID = vaultUUID
item.ID = uuid
return &item, nil
item := item2.ToModel()
for k, v := range newData {
item.Fields = append(item.Fields, model.ItemField{Label: k, Value: v})
}
mockGetItemByIDFunc.Return(item, nil)
_, err := onePasswordItemReconciler.Reconcile(ctx, reconcile.Request{NamespacedName: key})
Expect(err).ToNot(HaveOccurred())
@@ -178,18 +163,11 @@ var _ = Describe("OnePasswordItem controller", func() {
"ice-cream-type": []byte(iceCream),
}
mocks.DoGetItemFunc = func(uuid string, vaultUUID string) (*onepassword.Item, error) {
item := onepassword.Item{}
item.Title = "!my sECReT it3m%"
item.Fields = []*onepassword.ItemField{}
for k, v := range testData {
item.Fields = append(item.Fields, &onepassword.ItemField{Label: k, Value: v})
}
item.Version = item1.Version + 1
item.Vault.ID = vaultUUID
item.ID = uuid
return &item, nil
item := item2.ToModel()
for k, v := range testData {
item.Fields = append(item.Fields, model.ItemField{Label: k, Value: v})
}
mockGetItemByIDFunc.Return(item, nil)
By("Creating a new OnePasswordItem successfully")
Expect(k8sClient.Create(ctx, toCreate)).Should(Succeed())
@@ -326,6 +304,54 @@ var _ = Describe("OnePasswordItem controller", func() {
}, timeout, interval).Should(BeTrue())
Expect(secret.Type).Should(Equal(v1.SecretType(customType)))
})
It("Should handle 1Password Item with a file and populate secret correctly", func() {
ctx := context.Background()
spec := onepasswordv1.OnePasswordItemSpec{
ItemPath: item1.Path,
}
key := types.NamespacedName{
Name: "item-with-file",
Namespace: namespace,
}
toCreate := &onepasswordv1.OnePasswordItem{
ObjectMeta: metav1.ObjectMeta{
Name: key.Name,
Namespace: key.Namespace,
},
Spec: spec,
}
fileContent := []byte("dummy-cert-content")
item := item1.ToModel()
item.Files = []model.File{
{
ID: "file-id-123",
Name: "server.crt",
ContentPath: fmt.Sprintf("/v1/vaults/%s/items/%s/files/file-id-123/content", item.VaultID, item.ID),
},
}
item.Files[0].SetContent(fileContent)
mockGetItemByIDFunc.Return(item, nil)
mockGetItemByIDFunc.On("GetFileContent", item.VaultID, item.ID, "file-id-123").Return(fileContent, nil)
By("Creating a new OnePasswordItem with file successfully")
Expect(k8sClient.Create(ctx, toCreate)).Should(Succeed())
createdSecret := &v1.Secret{}
Eventually(func() bool {
err := k8sClient.Get(ctx, key, createdSecret)
if err != nil {
return false
}
return true
}, timeout, interval).Should(BeTrue())
Expect(createdSecret.Data).Should(HaveKeyWithValue("server.crt", fileContent))
})
})
Context("Unhappy path", func() {

View File

@@ -31,10 +31,9 @@ import (
"testing"
"time"
"github.com/1Password/onepassword-operator/pkg/mocks"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/stretchr/testify/mock"
"k8s.io/client-go/kubernetes/scheme"
"k8s.io/client-go/rest"
@@ -45,6 +44,8 @@ import (
"sigs.k8s.io/controller-runtime/pkg/log/zap"
onepasswordcomv1 "github.com/1Password/onepassword-operator/api/v1"
"github.com/1Password/onepassword-operator/pkg/mocks"
"github.com/1Password/onepassword-operator/pkg/onepassword/model"
//+kubebuilder:scaffold:imports
)
@@ -78,8 +79,11 @@ var (
cancel context.CancelFunc
onePasswordItemReconciler *OnePasswordItemReconciler
deploymentReconciler *DeploymentReconciler
mockGetItemByIDFunc *mock.Call
item1 = &TestItem{
ItemID: "nwrhuano7bcwddcviubpp4mhfq",
VaultID: "hfnjvi6aymbsnfc2xeeoheizda",
Name: "test-item",
Version: 123,
Path: "vaults/hfnjvi6aymbsnfc2xeeoheizda/items/nwrhuano7bcwddcviubpp4mhfq",
@@ -94,6 +98,8 @@ var (
}
item2 = &TestItem{
ItemID: "nwrhuano7bcwddcviubpp4mhf2",
VaultID: "hfnjvi6aymbsnfc2xeeoheizd2",
Name: "test-item2",
Path: "vaults/hfnjvi6aymbsnfc2xeeoheizd2/items/nwrhuano7bcwddcviubpp4mhf2",
Version: 456,
@@ -109,6 +115,8 @@ var (
)
type TestItem struct {
ItemID string
VaultID string
Name string
Version int
Path string
@@ -116,6 +124,20 @@ type TestItem struct {
SecretData map[string][]byte
}
func (ti *TestItem) ToModel() *model.Item {
item := &model.Item{}
item.Version = ti.Version
item.VaultID = ti.VaultID
item.ID = ti.ItemID
item.Fields = []model.ItemField{}
for k, v := range ti.Data {
item.Fields = append(item.Fields, model.ItemField{Label: k, Value: v})
}
return item
}
func TestAPIs(t *testing.T) {
RegisterFailHandler(Fail)
@@ -153,12 +175,13 @@ var _ = BeforeSuite(func() {
})
Expect(err).ToNot(HaveOccurred())
opConnectClient := &mocks.TestClient{}
mockOpClient := &mocks.TestClient{}
mockGetItemByIDFunc = mockOpClient.On("GetItemByID", mock.Anything, mock.Anything)
onePasswordItemReconciler = &OnePasswordItemReconciler{
Client: k8sManager.GetClient(),
Scheme: k8sManager.GetScheme(),
OpConnectClient: opConnectClient,
Client: k8sManager.GetClient(),
Scheme: k8sManager.GetScheme(),
OpClient: mockOpClient,
}
err = (onePasswordItemReconciler).SetupWithManager(k8sManager)
Expect(err).ToNot(HaveOccurred())
@@ -167,7 +190,7 @@ var _ = BeforeSuite(func() {
deploymentReconciler = &DeploymentReconciler{
Client: k8sManager.GetClient(),
Scheme: k8sManager.GetScheme(),
OpConnectClient: opConnectClient,
OpClient: mockOpClient,
OpAnnotationRegExp: r,
}
err = (deploymentReconciler).SetupWithManager(k8sManager)

View File

@@ -2,17 +2,13 @@ package kubernetessecrets
import (
"context"
errs "errors"
"fmt"
"reflect"
"regexp"
"strings"
"reflect"
errs "errors"
"github.com/1Password/connect-sdk-go/onepassword"
"github.com/1Password/onepassword-operator/pkg/onepassword/model"
"github.com/1Password/onepassword-operator/pkg/utils"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/errors"
@@ -34,11 +30,11 @@ var ErrCannotUpdateSecretType = errs.New("Cannot change secret type. Secret type
var log = logf.Log
func CreateKubernetesSecretFromItem(kubeClient kubernetesClient.Client, secretName, namespace string, item *onepassword.Item, autoRestart string, labels map[string]string, secretType string, ownerRef *metav1.OwnerReference) error {
func CreateKubernetesSecretFromItem(ctx context.Context, kubeClient kubernetesClient.Client, secretName, namespace string, item *model.Item, autoRestart string, labels map[string]string, secretType string, ownerRef *metav1.OwnerReference) error {
itemVersion := fmt.Sprint(item.Version)
secretAnnotations := map[string]string{
VersionAnnotation: itemVersion,
ItemPathAnnotation: fmt.Sprintf("vaults/%v/items/%v", item.Vault.ID, item.ID),
ItemPathAnnotation: fmt.Sprintf("vaults/%v/items/%v", item.VaultID, item.ID),
}
if autoRestart != "" {
@@ -53,10 +49,10 @@ func CreateKubernetesSecretFromItem(kubeClient kubernetesClient.Client, secretNa
secret := BuildKubernetesSecretFromOnePasswordItem(secretName, namespace, secretAnnotations, labels, secretType, *item, ownerRef)
currentSecret := &corev1.Secret{}
err := kubeClient.Get(context.Background(), types.NamespacedName{Name: secret.Name, Namespace: secret.Namespace}, currentSecret)
err := kubeClient.Get(ctx, types.NamespacedName{Name: secret.Name, Namespace: secret.Namespace}, currentSecret)
if err != nil && errors.IsNotFound(err) {
log.Info(fmt.Sprintf("Creating Secret %v at namespace '%v'", secret.Name, secret.Namespace))
return kubeClient.Create(context.Background(), secret)
return kubeClient.Create(ctx, secret)
} else if err != nil {
return err
}
@@ -82,7 +78,7 @@ func CreateKubernetesSecretFromItem(kubeClient kubernetesClient.Client, secretNa
currentSecret.ObjectMeta.Annotations = secretAnnotations
currentSecret.ObjectMeta.Labels = labels
currentSecret.Data = secret.Data
if err := kubeClient.Update(context.Background(), currentSecret); err != nil {
if err := kubeClient.Update(ctx, currentSecret); err != nil {
return fmt.Errorf("Kubernetes secret update failed: %w", err)
}
return nil
@@ -92,7 +88,7 @@ func CreateKubernetesSecretFromItem(kubeClient kubernetesClient.Client, secretNa
return nil
}
func BuildKubernetesSecretFromOnePasswordItem(name, namespace string, annotations map[string]string, labels map[string]string, secretType string, item onepassword.Item, ownerRef *metav1.OwnerReference) *corev1.Secret {
func BuildKubernetesSecretFromOnePasswordItem(name, namespace string, annotations map[string]string, labels map[string]string, secretType string, item model.Item, ownerRef *metav1.OwnerReference) *corev1.Secret {
var ownerRefs []metav1.OwnerReference
if ownerRef != nil {
ownerRefs = []metav1.OwnerReference{*ownerRef}
@@ -111,7 +107,7 @@ func BuildKubernetesSecretFromOnePasswordItem(name, namespace string, annotation
}
}
func BuildKubernetesSecretData(fields []*onepassword.ItemField, files []*onepassword.File) map[string][]byte {
func BuildKubernetesSecretData(fields []model.ItemField, files []model.File) map[string][]byte {
secretData := map[string][]byte{}
for i := 0; i < len(fields); i++ {
if fields[i].Value != "" {
@@ -124,7 +120,7 @@ func BuildKubernetesSecretData(fields []*onepassword.ItemField, files []*onepass
for _, file := range files {
content, err := file.Content()
if err != nil {
log.Error(err, "Could not load contents of file %s", file.Name)
log.Error(err, fmt.Sprintf("Could not load contents of file %s", file.Name))
continue
}
if content != nil {

View File

@@ -3,11 +3,10 @@ package kubernetessecrets
import (
"context"
"fmt"
"github.com/1Password/onepassword-operator/pkg/onepassword/model"
"strings"
"testing"
"github.com/1Password/connect-sdk-go/onepassword"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
@@ -18,25 +17,26 @@ import (
const restartDeploymentAnnotation = "false"
func TestCreateKubernetesSecretFromOnePasswordItem(t *testing.T) {
ctx := context.Background()
secretName := "test-secret-name"
namespace := "test"
item := onepassword.Item{}
item := model.Item{}
item.Fields = generateFields(5)
item.Version = 123
item.Vault.ID = "hfnjvi6aymbsnfc2xeeoheizda"
item.VaultID = "hfnjvi6aymbsnfc2xeeoheizda"
item.ID = "h46bb3jddvay7nxopfhvlwg35q"
kubeClient := fake.NewClientBuilder().Build()
secretLabels := map[string]string{}
secretType := ""
err := CreateKubernetesSecretFromItem(kubeClient, secretName, namespace, &item, restartDeploymentAnnotation, secretLabels, secretType, nil)
err := CreateKubernetesSecretFromItem(ctx, kubeClient, secretName, namespace, &item, restartDeploymentAnnotation, secretLabels, secretType, nil)
if err != nil {
t.Errorf("Unexpected error: %v", err)
}
createdSecret := &corev1.Secret{}
err = kubeClient.Get(context.Background(), types.NamespacedName{Name: secretName, Namespace: namespace}, createdSecret)
err = kubeClient.Get(ctx, types.NamespacedName{Name: secretName, Namespace: namespace}, createdSecret)
if err != nil {
t.Errorf("Secret was not created: %v", err)
@@ -46,13 +46,14 @@ func TestCreateKubernetesSecretFromOnePasswordItem(t *testing.T) {
}
func TestKubernetesSecretFromOnePasswordItemOwnerReferences(t *testing.T) {
ctx := context.Background()
secretName := "test-secret-name"
namespace := "test"
item := onepassword.Item{}
item := model.Item{}
item.Fields = generateFields(5)
item.Version = 123
item.Vault.ID = "hfnjvi6aymbsnfc2xeeoheizda"
item.VaultID = "hfnjvi6aymbsnfc2xeeoheizda"
item.ID = "h46bb3jddvay7nxopfhvlwg35q"
kubeClient := fake.NewClientBuilder().Build()
@@ -65,12 +66,12 @@ func TestKubernetesSecretFromOnePasswordItemOwnerReferences(t *testing.T) {
Name: "test-deployment",
UID: types.UID("test-uid"),
}
err := CreateKubernetesSecretFromItem(kubeClient, secretName, namespace, &item, restartDeploymentAnnotation, secretLabels, secretType, ownerRef)
err := CreateKubernetesSecretFromItem(ctx, kubeClient, secretName, namespace, &item, restartDeploymentAnnotation, secretLabels, secretType, ownerRef)
if err != nil {
t.Errorf("Unexpected error: %v", err)
}
createdSecret := &corev1.Secret{}
err = kubeClient.Get(context.Background(), types.NamespacedName{Name: secretName, Namespace: namespace}, createdSecret)
err = kubeClient.Get(ctx, types.NamespacedName{Name: secretName, Namespace: namespace}, createdSecret)
// Check owner references.
gotOwnerRefs := createdSecret.ObjectMeta.OwnerReferences
@@ -91,37 +92,38 @@ func TestKubernetesSecretFromOnePasswordItemOwnerReferences(t *testing.T) {
}
func TestUpdateKubernetesSecretFromOnePasswordItem(t *testing.T) {
ctx := context.Background()
secretName := "test-secret-update"
namespace := "test"
item := onepassword.Item{}
item := model.Item{}
item.Fields = generateFields(5)
item.Version = 123
item.Vault.ID = "hfnjvi6aymbsnfc2xeeoheizda"
item.VaultID = "hfnjvi6aymbsnfc2xeeoheizda"
item.ID = "h46bb3jddvay7nxopfhvlwg35q"
kubeClient := fake.NewClientBuilder().Build()
secretLabels := map[string]string{}
secretType := ""
err := CreateKubernetesSecretFromItem(kubeClient, secretName, namespace, &item, restartDeploymentAnnotation, secretLabels, secretType, nil)
err := CreateKubernetesSecretFromItem(ctx, kubeClient, secretName, namespace, &item, restartDeploymentAnnotation, secretLabels, secretType, nil)
if err != nil {
t.Errorf("Unexpected error: %v", err)
}
// Updating kubernetes secret with new item
newItem := onepassword.Item{}
newItem := model.Item{}
newItem.Fields = generateFields(6)
newItem.Version = 456
newItem.Vault.ID = "hfnjvi6aymbsnfc2xeeoheizda"
newItem.VaultID = "hfnjvi6aymbsnfc2xeeoheizda"
newItem.ID = "h46bb3jddvay7nxopfhvlwg35q"
err = CreateKubernetesSecretFromItem(kubeClient, secretName, namespace, &newItem, restartDeploymentAnnotation, secretLabels, secretType, nil)
err = CreateKubernetesSecretFromItem(ctx, kubeClient, secretName, namespace, &newItem, restartDeploymentAnnotation, secretLabels, secretType, nil)
if err != nil {
t.Errorf("Unexpected error: %v", err)
}
updatedSecret := &corev1.Secret{}
err = kubeClient.Get(context.Background(), types.NamespacedName{Name: secretName, Namespace: namespace}, updatedSecret)
err = kubeClient.Get(ctx, types.NamespacedName{Name: secretName, Namespace: namespace}, updatedSecret)
if err != nil {
t.Errorf("Secret was not found: %v", err)
@@ -147,7 +149,7 @@ func TestBuildKubernetesSecretFromOnePasswordItem(t *testing.T) {
annotations := map[string]string{
annotationKey: annotationValue,
}
item := onepassword.Item{}
item := model.Item{}
item.Fields = generateFields(5)
labels := map[string]string{}
secretType := ""
@@ -173,10 +175,10 @@ func TestBuildKubernetesSecretFixesInvalidLabels(t *testing.T) {
"annotationKey": "annotationValue",
}
labels := map[string]string{}
item := onepassword.Item{}
item := model.Item{}
secretType := ""
item.Fields = []*onepassword.ItemField{
item.Fields = []model.ItemField{
{
Label: "label w%th invalid ch!rs-",
Value: "value1",
@@ -206,25 +208,26 @@ func TestBuildKubernetesSecretFixesInvalidLabels(t *testing.T) {
}
func TestCreateKubernetesTLSSecretFromOnePasswordItem(t *testing.T) {
ctx := context.Background()
secretName := "tls-test-secret-name"
namespace := "test"
item := onepassword.Item{}
item := model.Item{}
item.Fields = generateFields(5)
item.Version = 123
item.Vault.ID = "hfnjvi6aymbsnfc2xeeoheizda"
item.VaultID = "hfnjvi6aymbsnfc2xeeoheizda"
item.ID = "h46bb3jddvay7nxopfhvlwg35q"
kubeClient := fake.NewClientBuilder().Build()
secretLabels := map[string]string{}
secretType := "kubernetes.io/tls"
err := CreateKubernetesSecretFromItem(kubeClient, secretName, namespace, &item, restartDeploymentAnnotation, secretLabels, secretType, nil)
err := CreateKubernetesSecretFromItem(ctx, kubeClient, secretName, namespace, &item, restartDeploymentAnnotation, secretLabels, secretType, nil)
if err != nil {
t.Errorf("Unexpected error: %v", err)
}
createdSecret := &corev1.Secret{}
err = kubeClient.Get(context.Background(), types.NamespacedName{Name: secretName, Namespace: namespace}, createdSecret)
err = kubeClient.Get(ctx, types.NamespacedName{Name: secretName, Namespace: namespace}, createdSecret)
if err != nil {
t.Errorf("Secret was not created: %v", err)
@@ -235,13 +238,13 @@ func TestCreateKubernetesTLSSecretFromOnePasswordItem(t *testing.T) {
}
}
func compareAnnotationsToItem(annotations map[string]string, item onepassword.Item, t *testing.T) {
func compareAnnotationsToItem(annotations map[string]string, item model.Item, t *testing.T) {
actualVaultId, actualItemId, err := ParseVaultIdAndItemIdFromPath(annotations[ItemPathAnnotation])
if err != nil {
t.Errorf("Was unable to parse Item Path")
}
if actualVaultId != item.Vault.ID {
t.Errorf("Expected annotation vault id to be %v but was %v", item.Vault.ID, actualVaultId)
if actualVaultId != item.VaultID {
t.Errorf("Expected annotation vault id to be %v but was %v", item.VaultID, actualVaultId)
}
if actualItemId != item.ID {
t.Errorf("Expected annotation item id to be %v but was %v", item.ID, actualItemId)
@@ -255,7 +258,7 @@ func compareAnnotationsToItem(annotations map[string]string, item onepassword.It
}
}
func compareFields(actualFields []*onepassword.ItemField, secretData map[string][]byte, t *testing.T) {
func compareFields(actualFields []model.ItemField, secretData map[string][]byte, t *testing.T) {
for i := 0; i < len(actualFields); i++ {
value, found := secretData[actualFields[i].Label]
if !found {
@@ -267,14 +270,13 @@ func compareFields(actualFields []*onepassword.ItemField, secretData map[string]
}
}
func generateFields(numToGenerate int) []*onepassword.ItemField {
fields := []*onepassword.ItemField{}
func generateFields(numToGenerate int) []model.ItemField {
fields := []model.ItemField{}
for i := 0; i < numToGenerate; i++ {
field := onepassword.ItemField{
fields = append(fields, model.ItemField{
Label: "key" + fmt.Sprint(i),
Value: "value" + fmt.Sprint(i),
}
fields = append(fields, &field)
})
}
return fields
}

View File

@@ -1,151 +1,38 @@
package mocks
import (
"github.com/1Password/connect-sdk-go/onepassword"
"context"
"github.com/stretchr/testify/mock"
"github.com/1Password/onepassword-operator/pkg/onepassword/model"
)
type TestClient struct {
GetVaultsFunc func() ([]onepassword.Vault, error)
GetVaultsByTitleFunc func(title string) ([]onepassword.Vault, error)
GetVaultFunc func(uuid string) (*onepassword.Vault, error)
GetVaultByUUIDFunc func(uuid string) (*onepassword.Vault, error)
GetVaultByTitleFunc func(title string) (*onepassword.Vault, error)
GetItemFunc func(itemQuery string, vaultQuery string) (*onepassword.Item, error)
GetItemByUUIDFunc func(uuid string, vaultQuery string) (*onepassword.Item, error)
GetItemByTitleFunc func(title string, vaultQuery string) (*onepassword.Item, error)
GetItemsFunc func(vaultQuery string) ([]onepassword.Item, error)
GetItemsByTitleFunc func(title string, vaultQuery string) ([]onepassword.Item, error)
CreateItemFunc func(item *onepassword.Item, vaultQuery string) (*onepassword.Item, error)
UpdateItemFunc func(item *onepassword.Item, vaultQuery string) (*onepassword.Item, error)
DeleteItemFunc func(item *onepassword.Item, vaultQuery string) error
DeleteItemByIDFunc func(itemUUID string, vaultQuery string) error
DeleteItemByTitleFunc func(title string, vaultQuery string) error
GetFilesFunc func(itemQuery string, vaultQuery string) ([]onepassword.File, error)
GetFileFunc func(uuid string, itemQuery string, vaultQuery string) (*onepassword.File, error)
GetFileContentFunc func(file *onepassword.File) ([]byte, error)
DownloadFileFunc func(file *onepassword.File, targetDirectory string, overwrite bool) (string, error)
LoadStructFromItemByUUIDFunc func(config interface{}, itemUUID string, vaultQuery string) error
LoadStructFromItemByTitleFunc func(config interface{}, itemTitle string, vaultQuery string) error
LoadStructFromItemFunc func(config interface{}, itemQuery string, vaultQuery string) error
LoadStructFunc func(config interface{}) error
mock.Mock
}
var (
DoGetVaultsFunc func() ([]onepassword.Vault, error)
DoGetVaultsByTitleFunc func(title string) ([]onepassword.Vault, error)
DoGetVaultFunc func(uuid string) (*onepassword.Vault, error)
DoGetVaultByUUIDFunc func(uuid string) (*onepassword.Vault, error)
DoGetVaultByTitleFunc func(title string) (*onepassword.Vault, error)
DoGetItemFunc func(itemQuery string, vaultQuery string) (*onepassword.Item, error)
DoGetItemByUUIDFunc func(uuid string, vaultQuery string) (*onepassword.Item, error)
DoGetItemByTitleFunc func(title string, vaultQuery string) (*onepassword.Item, error)
DoGetItemsFunc func(vaultQuery string) ([]onepassword.Item, error)
DoGetItemsByTitleFunc func(title string, vaultQuery string) ([]onepassword.Item, error)
DoCreateItemFunc func(item *onepassword.Item, vaultQuery string) (*onepassword.Item, error)
DoUpdateItemFunc func(item *onepassword.Item, vaultQuery string) (*onepassword.Item, error)
DoDeleteItemFunc func(item *onepassword.Item, vaultQuery string) error
DoDeleteItemByIDFunc func(itemUUID string, vaultQuery string) error
DoDeleteItemByTitleFunc func(title string, vaultQuery string) error
DoGetFilesFunc func(itemQuery string, vaultQuery string) ([]onepassword.File, error)
DoGetFileFunc func(uuid string, itemQuery string, vaultQuery string) (*onepassword.File, error)
DoGetFileContentFunc func(file *onepassword.File) ([]byte, error)
DoDownloadFileFunc func(file *onepassword.File, targetDirectory string, overwrite bool) (string, error)
DoLoadStructFromItemByUUIDFunc func(config interface{}, itemUUID string, vaultQuery string) error
DoLoadStructFromItemByTitleFunc func(config interface{}, itemTitle string, vaultQuery string) error
DoLoadStructFromItemFunc func(config interface{}, itemQuery string, vaultQuery string) error
DoLoadStructFunc func(config interface{}) error
)
// Do is the mock client's `Do` func
func (m *TestClient) GetVaults() ([]onepassword.Vault, error) {
return DoGetVaultsFunc()
func (tc *TestClient) GetItemByID(ctx context.Context, vaultID, itemID string) (*model.Item, error) {
args := tc.Called(vaultID, itemID)
if args.Get(0) == nil {
return nil, args.Error(1)
}
return args.Get(0).(*model.Item), args.Error(1)
}
func (m *TestClient) GetVaultsByTitle(title string) ([]onepassword.Vault, error) {
return DoGetVaultsByTitleFunc(title)
func (tc *TestClient) GetItemsByTitle(ctx context.Context, vaultID, itemTitle string) ([]model.Item, error) {
args := tc.Called(vaultID, itemTitle)
return args.Get(0).([]model.Item), args.Error(1)
}
func (m *TestClient) GetVault(vaultQuery string) (*onepassword.Vault, error) {
return DoGetVaultFunc(vaultQuery)
func (tc *TestClient) GetFileContent(ctx context.Context, vaultID, itemID, fileID string) ([]byte, error) {
args := tc.Called(vaultID, itemID, fileID)
if args.Get(0) == nil {
return nil, args.Error(1)
}
return args.Get(0).([]byte), args.Error(1)
}
func (m *TestClient) GetVaultByUUID(uuid string) (*onepassword.Vault, error) {
return DoGetVaultByUUIDFunc(uuid)
}
func (m *TestClient) GetVaultByTitle(title string) (*onepassword.Vault, error) {
return DoGetVaultByTitleFunc(title)
}
func (m *TestClient) GetItem(itemQuery string, vaultQuery string) (*onepassword.Item, error) {
return DoGetItemFunc(itemQuery, vaultQuery)
}
func (m *TestClient) GetItemByUUID(uuid string, vaultQuery string) (*onepassword.Item, error) {
return DoGetItemByUUIDFunc(uuid, vaultQuery)
}
func (m *TestClient) GetItemByTitle(title string, vaultQuery string) (*onepassword.Item, error) {
return DoGetItemByTitleFunc(title, vaultQuery)
}
func (m *TestClient) GetItems(vaultQuery string) ([]onepassword.Item, error) {
return DoGetItemsFunc(vaultQuery)
}
func (m *TestClient) GetItemsByTitle(title string, vaultQuery string) ([]onepassword.Item, error) {
return DoGetItemsByTitleFunc(title, vaultQuery)
}
func (m *TestClient) CreateItem(item *onepassword.Item, vaultQuery string) (*onepassword.Item, error) {
return DoCreateItemFunc(item, vaultQuery)
}
func (m *TestClient) UpdateItem(item *onepassword.Item, vaultQuery string) (*onepassword.Item, error) {
return DoUpdateItemFunc(item, vaultQuery)
}
func (m *TestClient) DeleteItem(item *onepassword.Item, vaultQuery string) error {
return DoDeleteItemFunc(item, vaultQuery)
}
func (m *TestClient) DeleteItemByID(itemUUID string, vaultQuery string) error {
return DoDeleteItemByIDFunc(itemUUID, vaultQuery)
}
func (m *TestClient) DeleteItemByTitle(title string, vaultQuery string) error {
return DoDeleteItemByTitleFunc(title, vaultQuery)
}
func (m *TestClient) GetFiles(itemQuery string, vaultQuery string) ([]onepassword.File, error) {
return DoGetFilesFunc(itemQuery, vaultQuery)
}
func (m *TestClient) GetFile(uuid string, itemQuery string, vaultQuery string) (*onepassword.File, error) {
return DoGetFileFunc(uuid, itemQuery, vaultQuery)
}
func (m *TestClient) GetFileContent(file *onepassword.File) ([]byte, error) {
return DoGetFileContentFunc(file)
}
func (m *TestClient) DownloadFile(file *onepassword.File, targetDirectory string, overwrite bool) (string, error) {
return DoDownloadFileFunc(file, targetDirectory, overwrite)
}
func (m *TestClient) LoadStructFromItemByUUID(config interface{}, itemUUID string, vaultQuery string) error {
return DoLoadStructFromItemByUUIDFunc(config, itemUUID, vaultQuery)
}
func (m *TestClient) LoadStructFromItemByTitle(config interface{}, itemTitle string, vaultQuery string) error {
return DoLoadStructFromItemByTitleFunc(config, itemTitle, vaultQuery)
}
func (m *TestClient) LoadStructFromItem(config interface{}, itemQuery string, vaultQuery string) error {
return DoLoadStructFromItemFunc(config, itemQuery, vaultQuery)
}
func (m *TestClient) LoadStruct(config interface{}) error {
return DoLoadStructFunc(config)
func (tc *TestClient) GetVaultsByTitle(ctx context.Context, title string) ([]model.Vault, error) {
args := tc.Called(title)
return args.Get(0).([]model.Vault), args.Error(1)
}

View File

@@ -0,0 +1,56 @@
package client
import (
"context"
"errors"
"os"
"github.com/go-logr/logr"
"github.com/1Password/onepassword-operator/pkg/onepassword/client/connect"
"github.com/1Password/onepassword-operator/pkg/onepassword/client/sdk"
"github.com/1Password/onepassword-operator/pkg/onepassword/model"
)
// Client is an interface for interacting with 1Password items and vaults.
type Client interface {
GetItemByID(ctx context.Context, vaultID, itemID string) (*model.Item, error)
GetItemsByTitle(ctx context.Context, vaultID, itemTitle string) ([]model.Item, error)
GetFileContent(ctx context.Context, vaultID, itemID, fileID string) ([]byte, error)
GetVaultsByTitle(ctx context.Context, title string) ([]model.Vault, error)
}
type Config struct {
Logger logr.Logger
Version string
}
// NewFromEnvironment creates a new 1Password client based on the provided configuration.
func NewFromEnvironment(ctx context.Context, cfg Config) (Client, error) {
connectHost, _ := os.LookupEnv("OP_CONNECT_HOST")
connectToken, _ := os.LookupEnv("OP_CONNECT_TOKEN")
serviceAccountToken, _ := os.LookupEnv("OP_SERVICE_ACCOUNT_TOKEN")
if connectHost != "" && connectToken != "" && serviceAccountToken != "" {
return nil, errors.New("invalid configuration. Either Connect or Service Account credentials should be set, not both")
}
if serviceAccountToken != "" {
cfg.Logger.Info("Using Service Account Token")
return sdk.NewClient(ctx, sdk.Config{
ServiceAccountToken: serviceAccountToken,
IntegrationName: "1password-operator",
IntegrationVersion: cfg.Version,
})
}
if connectHost != "" && connectToken != "" {
cfg.Logger.Info("Using 1Password Connect")
return connect.NewClient(connect.Config{
ConnectHost: connectHost,
ConnectToken: connectToken,
}), nil
}
return nil, errors.New("invalid configuration. Connect or Service Account credentials should be set")
}

View File

@@ -0,0 +1,103 @@
package connect
import (
"context"
"errors"
"fmt"
"time"
"github.com/1Password/connect-sdk-go/connect"
"github.com/1Password/connect-sdk-go/onepassword"
"github.com/1Password/onepassword-operator/pkg/onepassword/model"
)
// Config holds the configuration for the Connect client.
type Config struct {
ConnectHost string
ConnectToken string
}
// Connect is a client for interacting with 1Password using the Connect API.
type Connect struct {
client connect.Client
}
// NewClient creates a new Connect client using provided configuration.
func NewClient(config Config) *Connect {
return &Connect{
client: connect.NewClient(config.ConnectHost, config.ConnectToken),
}
}
func (c *Connect) GetItemByID(ctx context.Context, vaultID, itemID string) (*model.Item, error) {
connectItem, err := c.client.GetItemByUUID(itemID, vaultID)
if err != nil {
return nil, fmt.Errorf("failed to GetItemByID using 1Password Connect: %w", err)
}
var item model.Item
item.FromConnectItem(connectItem)
return &item, nil
}
func (c *Connect) GetItemsByTitle(ctx context.Context, vaultID, itemTitle string) ([]model.Item, error) {
// Get all items in the vault with the specified title
connectItems, err := c.client.GetItemsByTitle(itemTitle, vaultID)
if err != nil {
return nil, fmt.Errorf("failed to GetItemsByTitle using 1Password Connect: %w", err)
}
items := make([]model.Item, len(connectItems))
for i, connectItem := range connectItems {
var item model.Item
item.FromConnectItem(&connectItem)
items[i] = item
}
return items, nil
}
// GetFileContent retrieves the content of a file from a 1Password item.
// As the Connect has a delay when synchronizing files and returns a 500 error in this case, this function implements a retry mechanism.
func (c *Connect) GetFileContent(ctx context.Context, vaultID, itemID, fileID string) ([]byte, error) {
const maxRetries = 5
const delay = 1 * time.Second
var lastErr error
for i := 0; i < maxRetries; i++ {
bytes, err := c.client.GetFileContent(&onepassword.File{
ContentPath: fmt.Sprintf("/v1/vaults/%s/items/%s/files/%s/content", vaultID, itemID, fileID),
})
if err == nil {
return bytes, nil
}
var connectErr *onepassword.Error
if errors.As(err, &connectErr) && connectErr.StatusCode == 500 {
lastErr = err
time.Sleep(delay)
continue
}
return nil, fmt.Errorf("failed to GetFileContent using 1Password Connect: %w", err)
}
return nil, fmt.Errorf("failed to GetFileContent using 1Password Connect after %d retries: %w", maxRetries, lastErr)
}
func (c *Connect) GetVaultsByTitle(ctx context.Context, vaultQuery string) ([]model.Vault, error) {
connectVaults, err := c.client.GetVaultsByTitle(vaultQuery)
if err != nil {
return nil, fmt.Errorf("failed to GetVaultsByTitle using 1Password Connect: %w", err)
}
var vaults []model.Vault
for _, connectVault := range connectVaults {
if vaultQuery == connectVault.Name {
var vault model.Vault
vault.FromConnectVault(&connectVault)
vaults = append(vaults, vault)
}
}
return vaults, nil
}

View File

@@ -0,0 +1,241 @@
package connect
import (
"context"
"errors"
"testing"
"time"
"github.com/stretchr/testify/require"
"github.com/1Password/connect-sdk-go/onepassword"
clienttesting "github.com/1Password/onepassword-operator/pkg/onepassword/client/testing"
"github.com/1Password/onepassword-operator/pkg/onepassword/client/testing/mock"
"github.com/1Password/onepassword-operator/pkg/onepassword/model"
)
const VaultTitleEmployee = "Employee"
func TestConnect_GetItemByID(t *testing.T) {
connectItem := clienttesting.CreateConnectItem()
testCases := map[string]struct {
mockClient func() *mock.ConnectClientMock
check func(t *testing.T, item *model.Item, err error)
}{
"should return an item": {
mockClient: func() *mock.ConnectClientMock {
mockConnectClient := &mock.ConnectClientMock{}
mockConnectClient.On("GetItemByUUID", "item-id", "vault-id").Return(connectItem, nil)
return mockConnectClient
},
check: func(t *testing.T, item *model.Item, err error) {
require.NoError(t, err)
clienttesting.CheckConnectItemMapping(t, connectItem, item)
},
},
"should return an error": {
mockClient: func() *mock.ConnectClientMock {
mockConnectClient := &mock.ConnectClientMock{}
mockConnectClient.On("GetItemByUUID", "item-id", "vault-id").Return((*onepassword.Item)(nil), errors.New("error"))
return mockConnectClient
},
check: func(t *testing.T, item *model.Item, err error) {
require.Error(t, err)
require.Nil(t, item)
},
},
}
for description, tc := range testCases {
t.Run(description, func(t *testing.T) {
client := &Connect{client: tc.mockClient()}
item, err := client.GetItemByID(context.Background(), "vault-id", "item-id")
tc.check(t, item, err)
})
}
}
func TestConnect_GetItemsByTitle(t *testing.T) {
connectItem1 := clienttesting.CreateConnectItem()
connectItem2 := clienttesting.CreateConnectItem()
testCases := map[string]struct {
mockClient func() *mock.ConnectClientMock
check func(t *testing.T, items []model.Item, err error)
}{
"should return a single item": {
mockClient: func() *mock.ConnectClientMock {
mockConnectClient := &mock.ConnectClientMock{}
mockConnectClient.On("GetItemsByTitle", "item-title", "vault-id").Return(
[]onepassword.Item{
*connectItem1,
}, nil)
return mockConnectClient
},
check: func(t *testing.T, items []model.Item, err error) {
require.NoError(t, err)
require.Len(t, items, 1)
require.Equal(t, connectItem1.ID, items[0].ID)
},
},
"should return two items": {
mockClient: func() *mock.ConnectClientMock {
mockConnectClient := &mock.ConnectClientMock{}
mockConnectClient.On("GetItemsByTitle", "item-title", "vault-id").Return(
[]onepassword.Item{
*connectItem1,
*connectItem2,
}, nil)
return mockConnectClient
},
check: func(t *testing.T, items []model.Item, err error) {
require.NoError(t, err)
require.Len(t, items, 2)
clienttesting.CheckConnectItemMapping(t, connectItem1, &items[0])
clienttesting.CheckConnectItemMapping(t, connectItem2, &items[1])
},
},
"should return an error": {
mockClient: func() *mock.ConnectClientMock {
mockConnectClient := &mock.ConnectClientMock{}
mockConnectClient.On("GetItemsByTitle", "item-title", "vault-id").Return([]onepassword.Item{}, errors.New("error"))
return mockConnectClient
},
check: func(t *testing.T, items []model.Item, err error) {
require.Error(t, err)
require.Nil(t, items)
},
},
}
for description, tc := range testCases {
t.Run(description, func(t *testing.T) {
client := &Connect{client: tc.mockClient()}
items, err := client.GetItemsByTitle(context.Background(), "vault-id", "item-title")
tc.check(t, items, err)
})
}
}
func TestConnect_GetFileContent(t *testing.T) {
testCases := map[string]struct {
mockClient func() *mock.ConnectClientMock
check func(t *testing.T, content []byte, err error)
}{
"should return file content": {
mockClient: func() *mock.ConnectClientMock {
mockConnectClient := &mock.ConnectClientMock{}
mockConnectClient.On("GetFileContent", &onepassword.File{
ContentPath: "/v1/vaults/vault-id/items/item-id/files/file-id/content",
}).Return([]byte("file content"), nil)
return mockConnectClient
},
check: func(t *testing.T, content []byte, err error) {
require.NoError(t, err)
require.Equal(t, []byte("file content"), content)
},
},
"should return an error": {
mockClient: func() *mock.ConnectClientMock {
mockConnectClient := &mock.ConnectClientMock{}
mockConnectClient.On("GetFileContent", &onepassword.File{
ContentPath: "/v1/vaults/vault-id/items/item-id/files/file-id/content",
}).Return(nil, errors.New("error"))
return mockConnectClient
},
check: func(t *testing.T, content []byte, err error) {
require.Error(t, err)
require.Nil(t, content)
},
},
}
for description, tc := range testCases {
t.Run(description, func(t *testing.T) {
client := &Connect{client: tc.mockClient()}
content, err := client.GetFileContent(context.Background(), "vault-id", "item-id", "file-id")
tc.check(t, content, err)
})
}
}
func TestConnect_GetVaultsByTitle(t *testing.T) {
now := time.Now()
testCases := map[string]struct {
mockClient func() *mock.ConnectClientMock
check func(t *testing.T, vaults []model.Vault, err error)
}{
"should return a single vault": {
mockClient: func() *mock.ConnectClientMock {
mockConnectClient := &mock.ConnectClientMock{}
mockConnectClient.On("GetVaultsByTitle", VaultTitleEmployee).Return([]onepassword.Vault{
{
ID: "test-id",
Name: VaultTitleEmployee,
CreatedAt: now,
},
{
ID: "test-id-2",
Name: "Some other vault",
CreatedAt: now,
},
}, nil)
return mockConnectClient
},
check: func(t *testing.T, vaults []model.Vault, err error) {
require.NoError(t, err)
require.Len(t, vaults, 1)
require.Equal(t, "test-id", vaults[0].ID)
require.Equal(t, now, vaults[0].CreatedAt)
},
},
"should return a two vaults": {
mockClient: func() *mock.ConnectClientMock {
mockConnectClient := &mock.ConnectClientMock{}
mockConnectClient.On("GetVaultsByTitle", VaultTitleEmployee).Return([]onepassword.Vault{
{
ID: "test-id",
Name: VaultTitleEmployee,
CreatedAt: now,
},
{
ID: "test-id-2",
Name: VaultTitleEmployee,
CreatedAt: now,
},
}, nil)
return mockConnectClient
},
check: func(t *testing.T, vaults []model.Vault, err error) {
require.NoError(t, err)
require.Len(t, vaults, 2)
// Check the first vault
require.Equal(t, "test-id", vaults[0].ID)
require.Equal(t, now, vaults[0].CreatedAt)
// Check the second vault
require.Equal(t, "test-id-2", vaults[1].ID)
require.Equal(t, now, vaults[1].CreatedAt)
},
},
"should return an error": {
mockClient: func() *mock.ConnectClientMock {
mockConnectClient := &mock.ConnectClientMock{}
mockConnectClient.On("GetVaultsByTitle", VaultTitleEmployee).Return([]onepassword.Vault{}, errors.New("error"))
return mockConnectClient
},
check: func(t *testing.T, vaults []model.Vault, err error) {
require.Error(t, err)
require.Empty(t, vaults)
},
},
}
for description, tc := range testCases {
t.Run(description, func(t *testing.T) {
client := &Connect{client: tc.mockClient()}
vault, err := client.GetVaultsByTitle(context.Background(), VaultTitleEmployee)
tc.check(t, vault, err)
})
}
}

View File

@@ -0,0 +1,97 @@
package sdk
import (
"context"
"fmt"
"github.com/1Password/onepassword-operator/pkg/onepassword/model"
sdk "github.com/1password/onepassword-sdk-go"
)
// Config holds the configuration for the 1Password SDK client.
type Config struct {
ServiceAccountToken string
IntegrationName string
IntegrationVersion string
}
// SDK is a client for interacting with 1Password using the SDK.
type SDK struct {
client *sdk.Client
}
func NewClient(ctx context.Context, config Config) (*SDK, error) {
client, err := sdk.NewClient(ctx,
sdk.WithServiceAccountToken(config.ServiceAccountToken),
sdk.WithIntegrationInfo(config.IntegrationName, config.IntegrationVersion),
)
if err != nil {
return nil, fmt.Errorf("1Password sdk error: %w", err)
}
return &SDK{
client: client,
}, nil
}
func (s *SDK) GetItemByID(ctx context.Context, vaultID, itemID string) (*model.Item, error) {
sdkItem, err := s.client.Items().Get(ctx, vaultID, itemID)
if err != nil {
return nil, fmt.Errorf("failed to GetItemsByTitle using 1Password SDK: %w", err)
}
var item model.Item
item.FromSDKItem(&sdkItem)
return &item, nil
}
func (s *SDK) GetItemsByTitle(ctx context.Context, vaultID, itemTitle string) ([]model.Item, error) {
// Get all items in the vault
sdkItems, err := s.client.Items().List(ctx, vaultID)
if err != nil {
return nil, fmt.Errorf("failed to GetItemsByTitle using 1Password SDK: %w", err)
}
// Filter items by title
var items []model.Item
for _, sdkItem := range sdkItems {
if sdkItem.Title == itemTitle {
var item model.Item
item.FromSDKItemOverview(&sdkItem)
items = append(items, item)
}
}
return items, nil
}
func (s *SDK) GetFileContent(ctx context.Context, vaultID, itemID, fileID string) ([]byte, error) {
bytes, err := s.client.Items().Files().Read(ctx, vaultID, itemID, sdk.FileAttributes{
ID: fileID,
})
if err != nil {
return nil, fmt.Errorf("failed to GetFileContent using 1Password SDK: %w", err)
}
return bytes, nil
}
func (s *SDK) GetVaultsByTitle(ctx context.Context, title string) ([]model.Vault, error) {
// List all vaults
sdkVaults, err := s.client.Vaults().List(ctx)
if err != nil {
return nil, fmt.Errorf("failed to GetVaultsByTitle using 1Password SDK: %w", err)
}
// Filter vaults by title
var vaults []model.Vault
for _, sdkVault := range sdkVaults {
if sdkVault.Title == title {
var vault model.Vault
vault.FromSDKVault(&sdkVault)
vaults = append(vaults, vault)
}
}
return vaults, nil
}

View File

@@ -0,0 +1,288 @@
package sdk
import (
"context"
"errors"
"testing"
"time"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
clienttesting "github.com/1Password/onepassword-operator/pkg/onepassword/client/testing"
clientmock "github.com/1Password/onepassword-operator/pkg/onepassword/client/testing/mock"
"github.com/1Password/onepassword-operator/pkg/onepassword/model"
sdk "github.com/1password/onepassword-sdk-go"
)
const VaultTitleEmployee = "Employee"
func TestSDK_GetItemByID(t *testing.T) {
sdkItem := clienttesting.CreateSDKItem()
testCases := map[string]struct {
mockItemAPI func() *clientmock.ItemAPIMock
check func(t *testing.T, item *model.Item, err error)
}{
"should return a single item": {
mockItemAPI: func() *clientmock.ItemAPIMock {
m := &clientmock.ItemAPIMock{}
m.On("Get", context.Background(), "vault-id", "item-id").Return(*sdkItem, nil)
return m
},
check: func(t *testing.T, item *model.Item, err error) {
require.NoError(t, err)
clienttesting.CheckSDKItemMapping(t, sdkItem, item)
},
},
"should return an error": {
mockItemAPI: func() *clientmock.ItemAPIMock {
m := &clientmock.ItemAPIMock{}
m.On("Get", context.Background(), "vault-id", "item-id").Return(sdk.Item{}, errors.New("error"))
return m
},
check: func(t *testing.T, item *model.Item, err error) {
require.Error(t, err)
require.Empty(t, item)
},
},
}
for description, tc := range testCases {
t.Run(description, func(t *testing.T) {
client := &SDK{
client: &sdk.Client{
ItemsAPI: tc.mockItemAPI(),
},
}
item, err := client.GetItemByID(context.Background(), "vault-id", "item-id")
tc.check(t, item, err)
})
}
}
func TestSDK_GetItemsByTitle(t *testing.T) {
sdkItem1 := clienttesting.CreateSDKItemOverview()
sdkItem2 := clienttesting.CreateSDKItemOverview()
testCases := map[string]struct {
mockItemAPI func() *clientmock.ItemAPIMock
check func(t *testing.T, items []model.Item, err error)
}{
"should return a single item": {
mockItemAPI: func() *clientmock.ItemAPIMock {
m := &clientmock.ItemAPIMock{}
copySDKItem2 := *sdkItem2
copySDKItem2.Title = "Some other item"
m.On("List", context.Background(), "vault-id", mock.Anything).Return([]sdk.ItemOverview{
*sdkItem1,
copySDKItem2,
}, nil)
return m
},
check: func(t *testing.T, items []model.Item, err error) {
require.NoError(t, err)
require.Len(t, items, 1)
clienttesting.CheckSDKItemOverviewMapping(t, sdkItem1, &items[0])
},
},
"should return a two items": {
mockItemAPI: func() *clientmock.ItemAPIMock {
m := &clientmock.ItemAPIMock{}
m.On("List", context.Background(), "vault-id", mock.Anything).Return([]sdk.ItemOverview{
*sdkItem1,
*sdkItem2,
}, nil)
return m
},
check: func(t *testing.T, items []model.Item, err error) {
require.NoError(t, err)
require.Len(t, items, 2)
clienttesting.CheckSDKItemOverviewMapping(t, sdkItem1, &items[0])
clienttesting.CheckSDKItemOverviewMapping(t, sdkItem2, &items[1])
},
},
"should return empty list": {
mockItemAPI: func() *clientmock.ItemAPIMock {
m := &clientmock.ItemAPIMock{}
m.On("List", context.Background(), "vault-id", mock.Anything).Return([]sdk.ItemOverview{}, nil)
return m
},
check: func(t *testing.T, items []model.Item, err error) {
require.NoError(t, err)
require.Len(t, items, 0)
},
},
"should return an error": {
mockItemAPI: func() *clientmock.ItemAPIMock {
m := &clientmock.ItemAPIMock{}
m.On("List", context.Background(), "vault-id", mock.Anything).Return([]sdk.ItemOverview{}, errors.New("error"))
return m
},
check: func(t *testing.T, items []model.Item, err error) {
require.Error(t, err)
require.Empty(t, items)
},
},
}
for description, tc := range testCases {
t.Run(description, func(t *testing.T) {
client := &SDK{
client: &sdk.Client{
ItemsAPI: tc.mockItemAPI(),
},
}
items, err := client.GetItemsByTitle(context.Background(), "vault-id", "item-title")
tc.check(t, items, err)
})
}
}
func TestSDK_GetFileContent(t *testing.T) {
testCases := map[string]struct {
mockItemAPI func() *clientmock.ItemAPIMock
check func(t *testing.T, content []byte, err error)
}{
"should return file content": {
mockItemAPI: func() *clientmock.ItemAPIMock {
fileMock := &clientmock.FileAPIMock{}
fileMock.On("Read", mock.Anything, "vault-id", "item-id",
mock.MatchedBy(func(attr sdk.FileAttributes) bool {
return attr.ID == "file-id"
}),
).Return([]byte("file content"), nil)
itemMock := &clientmock.ItemAPIMock{
FilesAPI: fileMock,
}
itemMock.On("Files").Return(fileMock)
return itemMock
},
check: func(t *testing.T, content []byte, err error) {
require.NoError(t, err)
require.Equal(t, []byte("file content"), content)
},
},
"should return an error": {
mockItemAPI: func() *clientmock.ItemAPIMock {
fileMock := &clientmock.FileAPIMock{}
fileMock.On("Read", mock.Anything, "vault-id", "item-id",
mock.MatchedBy(func(attr sdk.FileAttributes) bool {
return attr.ID == "file-id"
}),
).Return(nil, errors.New("error"))
itemMock := &clientmock.ItemAPIMock{
FilesAPI: fileMock,
}
itemMock.On("Files").Return(fileMock)
return itemMock
},
check: func(t *testing.T, content []byte, err error) {
require.Error(t, err)
require.Nil(t, content)
},
},
}
for description, tc := range testCases {
t.Run(description, func(t *testing.T) {
client := &SDK{
client: &sdk.Client{
ItemsAPI: tc.mockItemAPI(),
},
}
content, err := client.GetFileContent(context.Background(), "vault-id", "item-id", "file-id")
tc.check(t, content, err)
})
}
}
func TestSDK_GetVaultsByTitle(t *testing.T) {
now := time.Now()
testCases := map[string]struct {
mockVaultAPI func() *clientmock.VaultAPIMock
check func(t *testing.T, vaults []model.Vault, err error)
}{
"should return a single vault": {
mockVaultAPI: func() *clientmock.VaultAPIMock {
m := &clientmock.VaultAPIMock{}
m.On("List", context.Background()).Return([]sdk.VaultOverview{
{
ID: "test-id",
Title: VaultTitleEmployee,
CreatedAt: now,
},
{
ID: "test-id-2",
Title: "Some other vault",
CreatedAt: now,
},
}, nil)
return m
},
check: func(t *testing.T, vaults []model.Vault, err error) {
require.NoError(t, err)
require.Len(t, vaults, 1)
require.Equal(t, "test-id", vaults[0].ID)
require.Equal(t, now, vaults[0].CreatedAt)
},
},
"should return a two vaults": {
mockVaultAPI: func() *clientmock.VaultAPIMock {
m := &clientmock.VaultAPIMock{}
m.On("List", context.Background()).Return([]sdk.VaultOverview{
{
ID: "test-id",
Title: VaultTitleEmployee,
CreatedAt: now,
},
{
ID: "test-id-2",
Title: VaultTitleEmployee,
CreatedAt: now,
},
}, nil)
return m
},
check: func(t *testing.T, vaults []model.Vault, err error) {
require.NoError(t, err)
require.Len(t, vaults, 2)
// Check the first vault
require.Equal(t, "test-id", vaults[0].ID)
require.Equal(t, now, vaults[0].CreatedAt)
// Check the second vault
require.Equal(t, "test-id-2", vaults[1].ID)
require.Equal(t, now, vaults[1].CreatedAt)
},
},
"should return an error": {
mockVaultAPI: func() *clientmock.VaultAPIMock {
m := &clientmock.VaultAPIMock{}
m.On("List", context.Background()).Return([]sdk.VaultOverview{}, errors.New("error"))
return m
},
check: func(t *testing.T, vaults []model.Vault, err error) {
require.Error(t, err)
require.Empty(t, vaults)
},
},
}
for description, tc := range testCases {
t.Run(description, func(t *testing.T) {
client := &SDK{
client: &sdk.Client{
VaultsAPI: tc.mockVaultAPI(),
},
}
vault, err := client.GetVaultsByTitle(context.Background(), VaultTitleEmployee)
tc.check(t, vault, err)
})
}
}

View File

@@ -0,0 +1,110 @@
package testing
import (
sdk "github.com/1password/onepassword-sdk-go"
"testing"
"time"
"github.com/stretchr/testify/require"
"github.com/1Password/connect-sdk-go/onepassword"
"github.com/1Password/onepassword-operator/pkg/onepassword/model"
)
func CreateConnectItem() *onepassword.Item {
return &onepassword.Item{
ID: "test-id",
Vault: onepassword.ItemVault{ID: "test-vault-id"},
Version: 1,
Tags: []string{"tag1", "tag2"},
Fields: []*onepassword.ItemField{
{Label: "label1", Value: "value1"},
{Label: "label2", Value: "value2"},
},
Files: []*onepassword.File{
{ID: "file-id-1", Name: "file1.txt", Size: 1234},
{ID: "file-id-2", Name: "file2.txt", Size: 1234},
},
}
}
func CreateSDKItem() *sdk.Item {
return &sdk.Item{
ID: "test-id",
VaultID: "test-vault-id",
Version: 1,
Tags: []string{"tag1", "tag2"},
Fields: []sdk.ItemField{
{Title: "label1", Value: "value1"},
{Title: "label2", Value: "value2"},
},
Files: []sdk.ItemFile{
{Attributes: sdk.FileAttributes{ID: "file-id-1", Name: "file1.txt", Size: 1234}},
{Attributes: sdk.FileAttributes{ID: "file-id-2", Name: "file2.txt", Size: 1234}},
},
CreatedAt: time.Now(),
}
}
func CreateSDKItemOverview() *sdk.ItemOverview {
return &sdk.ItemOverview{
ID: "test-id",
Title: "item-title",
VaultID: "test-vault-id",
Tags: []string{"tag1", "tag2"},
CreatedAt: time.Now(),
}
}
func CheckConnectItemMapping(t *testing.T, expected *onepassword.Item, actual *model.Item) {
t.Helper()
require.Equal(t, expected.ID, actual.ID)
require.Equal(t, expected.Vault.ID, actual.VaultID)
require.Equal(t, expected.Version, actual.Version)
require.ElementsMatch(t, expected.Tags, actual.Tags)
for i, field := range expected.Fields {
require.Equal(t, field.Label, actual.Fields[i].Label)
require.Equal(t, field.Value, actual.Fields[i].Value)
}
for i, file := range expected.Files {
require.Equal(t, file.ID, actual.Files[i].ID)
require.Equal(t, file.Name, actual.Files[i].Name)
require.Equal(t, file.Size, actual.Files[i].Size)
}
require.Equal(t, expected.CreatedAt, actual.CreatedAt)
}
func CheckSDKItemMapping(t *testing.T, expected *sdk.Item, actual *model.Item) {
t.Helper()
require.Equal(t, expected.ID, actual.ID)
require.Equal(t, expected.VaultID, actual.VaultID)
require.Equal(t, int(expected.Version), actual.Version)
require.ElementsMatch(t, expected.Tags, actual.Tags)
for i, field := range expected.Fields {
require.Equal(t, field.Title, actual.Fields[i].Label)
require.Equal(t, field.Value, actual.Fields[i].Value)
}
for i, file := range expected.Files {
require.Equal(t, file.Attributes.ID, actual.Files[i].ID)
require.Equal(t, file.Attributes.Name, actual.Files[i].Name)
require.Equal(t, int(file.Attributes.Size), actual.Files[i].Size)
}
require.Equal(t, expected.CreatedAt, actual.CreatedAt)
}
func CheckSDKItemOverviewMapping(t *testing.T, expected *sdk.ItemOverview, actual *model.Item) {
t.Helper()
require.Equal(t, expected.ID, actual.ID)
require.Equal(t, expected.VaultID, actual.VaultID)
require.ElementsMatch(t, expected.Tags, actual.Tags)
require.Equal(t, expected.CreatedAt, actual.CreatedAt)
}

View File

@@ -0,0 +1,130 @@
package mock
import (
"github.com/stretchr/testify/mock"
"github.com/1Password/connect-sdk-go/onepassword"
)
// ConnectClientMock is a mock implementation of the ConnectClient interface
type ConnectClientMock struct {
mock.Mock
}
func (c *ConnectClientMock) GetVaults() ([]onepassword.Vault, error) {
// Only implement this if mocking is needed
panic("implement me")
}
func (c *ConnectClientMock) GetVault(uuid string) (*onepassword.Vault, error) {
args := c.Called(uuid)
return args.Get(0).(*onepassword.Vault), args.Error(1)
}
func (c *ConnectClientMock) GetVaultByUUID(uuid string) (*onepassword.Vault, error) {
// Only implement this if mocking is needed
panic("implement me")
}
func (c *ConnectClientMock) GetVaultByTitle(title string) (*onepassword.Vault, error) {
// Only implement this if mocking is needed
panic("implement me")
}
func (c *ConnectClientMock) GetVaultsByTitle(title string) ([]onepassword.Vault, error) {
args := c.Called(title)
return args.Get(0).([]onepassword.Vault), args.Error(1)
}
func (c *ConnectClientMock) GetItems(vaultQuery string) ([]onepassword.Item, error) {
// Only implement this if mocking is needed
panic("implement me")
}
func (c *ConnectClientMock) GetItem(itemQuery, vaultQuery string) (*onepassword.Item, error) {
// Only implement this if mocking is needed
panic("implement me")
}
func (c *ConnectClientMock) GetItemByUUID(uuid string, vaultQuery string) (*onepassword.Item, error) {
args := c.Called(uuid, vaultQuery)
return args.Get(0).(*onepassword.Item), args.Error(1)
}
func (c *ConnectClientMock) GetItemByTitle(title string, vaultQuery string) (*onepassword.Item, error) {
// Only implement this if mocking is needed
panic("implement me")
}
func (c *ConnectClientMock) GetItemsByTitle(title string, vaultQuery string) ([]onepassword.Item, error) {
args := c.Called(title, vaultQuery)
return args.Get(0).([]onepassword.Item), args.Error(1)
}
func (c *ConnectClientMock) CreateItem(item *onepassword.Item, vaultQuery string) (*onepassword.Item, error) {
// Only implement this if mocking is needed
panic("implement me")
}
func (c *ConnectClientMock) UpdateItem(item *onepassword.Item, vaultQuery string) (*onepassword.Item, error) {
// Only implement this if mocking is needed
panic("implement me")
}
func (c *ConnectClientMock) DeleteItem(item *onepassword.Item, vaultQuery string) error {
// Only implement this if mocking is needed
panic("implement me")
}
func (c *ConnectClientMock) DeleteItemByID(itemUUID string, vaultQuery string) error {
// Only implement this if mocking is needed
panic("implement me")
}
func (c *ConnectClientMock) DeleteItemByTitle(title string, vaultQuery string) error {
// Only implement this if mocking is needed
panic("implement me")
}
func (c *ConnectClientMock) GetFiles(itemQuery string, vaultQuery string) ([]onepassword.File, error) {
// Only implement this if mocking is needed
panic("implement me")
}
func (c *ConnectClientMock) GetFile(uuid string, itemQuery string, vaultQuery string) (*onepassword.File, error) {
// Only implement this if mocking is needed
panic("implement me")
}
func (c *ConnectClientMock) GetFileContent(file *onepassword.File) ([]byte, error) {
args := c.Called(file)
if args.Get(0) == nil {
return nil, args.Error(1)
}
return args.Get(0).([]byte), args.Error(1)
}
func (c *ConnectClientMock) DownloadFile(file *onepassword.File, targetDirectory string, overwrite bool) (string, error) {
// Only implement this if mocking is needed
panic("implement me")
}
func (c *ConnectClientMock) LoadStructFromItemByUUID(config interface{}, itemUUID string, vaultQuery string) error {
// Only implement this if mocking is needed
panic("implement me")
}
func (c *ConnectClientMock) LoadStructFromItemByTitle(config interface{}, itemTitle string, vaultQuery string) error {
// Only implement this if mocking is needed
panic("implement me")
}
func (c *ConnectClientMock) LoadStructFromItem(config interface{}, itemQuery string, vaultQuery string) error {
// Only implement this if mocking is needed
panic("implement me")
}
func (c *ConnectClientMock) LoadStruct(config interface{}) error {
// Only implement this if mocking is needed
panic("implement me")
}

View File

@@ -0,0 +1,89 @@
package mock
import (
"context"
"github.com/stretchr/testify/mock"
sdk "github.com/1password/onepassword-sdk-go"
)
type VaultAPIMock struct {
mock.Mock
}
func (v *VaultAPIMock) List(ctx context.Context) ([]sdk.VaultOverview, error) {
args := v.Called(ctx)
return args.Get(0).([]sdk.VaultOverview), args.Error(1)
}
type ItemAPIMock struct {
mock.Mock
FilesAPI sdk.ItemsFilesAPI
}
func (i *ItemAPIMock) Create(ctx context.Context, params sdk.ItemCreateParams) (sdk.Item, error) {
//TODO implement me
panic("implement me")
}
func (i *ItemAPIMock) Get(ctx context.Context, vaultID string, itemID string) (sdk.Item, error) {
args := i.Called(ctx, vaultID, itemID)
return args.Get(0).(sdk.Item), args.Error(1)
}
func (i *ItemAPIMock) Put(ctx context.Context, item sdk.Item) (sdk.Item, error) {
//TODO implement me
panic("implement me")
}
func (i *ItemAPIMock) Delete(ctx context.Context, vaultID string, itemID string) error {
//TODO implement me
panic("implement me")
}
func (i *ItemAPIMock) Archive(ctx context.Context, vaultID string, itemID string) error {
//TODO implement me
panic("implement me")
}
func (i *ItemAPIMock) List(ctx context.Context, vaultID string, filters ...sdk.ItemListFilter) ([]sdk.ItemOverview, error) {
args := i.Called(ctx, vaultID, filters)
return args.Get(0).([]sdk.ItemOverview), args.Error(1)
}
func (i *ItemAPIMock) Shares() sdk.ItemsSharesAPI {
//TODO implement me
panic("implement me")
}
func (i *ItemAPIMock) Files() sdk.ItemsFilesAPI {
return i.FilesAPI
}
type FileAPIMock struct {
mock.Mock
}
func (f *FileAPIMock) Attach(ctx context.Context, item sdk.Item, fileParams sdk.FileCreateParams) (sdk.Item, error) {
//TODO implement me
panic("implement me")
}
func (f *FileAPIMock) Delete(ctx context.Context, item sdk.Item, sectionID string, fieldID string) (sdk.Item, error) {
//TODO implement me
panic("implement me")
}
func (f *FileAPIMock) ReplaceDocument(ctx context.Context, item sdk.Item, docParams sdk.DocumentCreateParams) (sdk.Item, error) {
//TODO implement me
panic("implement me")
}
func (f *FileAPIMock) Read(ctx context.Context, vaultID, itemID string, attributes sdk.FileAttributes) ([]byte, error) {
args := f.Called(ctx, vaultID, itemID, attributes)
if args.Get(0) == nil {
return nil, args.Error(1)
}
return args.Get(0).([]byte), args.Error(1)
}

View File

@@ -18,13 +18,13 @@ var logConnectSetup = logf.Log.WithName("ConnectSetup")
var deploymentPath = "../config/connect/deployment.yaml"
var servicePath = "../config/connect/service.yaml"
func SetupConnect(kubeClient client.Client, deploymentNamespace string) error {
err := setupService(kubeClient, servicePath, deploymentNamespace)
func SetupConnect(ctx context.Context, kubeClient client.Client, deploymentNamespace string) error {
err := setupService(ctx, kubeClient, servicePath, deploymentNamespace)
if err != nil {
return err
}
err = setupDeployment(kubeClient, deploymentPath, deploymentNamespace)
err = setupDeployment(ctx, kubeClient, deploymentPath, deploymentNamespace)
if err != nil {
return err
}
@@ -32,27 +32,27 @@ func SetupConnect(kubeClient client.Client, deploymentNamespace string) error {
return nil
}
func setupDeployment(kubeClient client.Client, deploymentPath string, deploymentNamespace string) error {
func setupDeployment(ctx context.Context, kubeClient client.Client, deploymentPath string, deploymentNamespace string) error {
existingDeployment := &appsv1.Deployment{}
// check if deployment has already been created
err := kubeClient.Get(context.Background(), types.NamespacedName{Name: "onepassword-connect", Namespace: deploymentNamespace}, existingDeployment)
err := kubeClient.Get(ctx, types.NamespacedName{Name: "onepassword-connect", Namespace: deploymentNamespace}, existingDeployment)
if err != nil {
if errors.IsNotFound(err) {
logConnectSetup.Info("No existing Connect deployment found. Creating Deployment")
return createDeployment(kubeClient, deploymentPath, deploymentNamespace)
return createDeployment(ctx, kubeClient, deploymentPath, deploymentNamespace)
}
}
return err
}
func createDeployment(kubeClient client.Client, deploymentPath string, deploymentNamespace string) error {
func createDeployment(ctx context.Context, kubeClient client.Client, deploymentPath string, deploymentNamespace string) error {
deployment, err := getDeploymentToCreate(deploymentPath, deploymentNamespace)
if err != nil {
return err
}
err = kubeClient.Create(context.Background(), deployment)
err = kubeClient.Create(ctx, deployment)
if err != nil {
return err
}
@@ -78,21 +78,21 @@ func getDeploymentToCreate(deploymentPath string, deploymentNamespace string) (*
return deployment, nil
}
func setupService(kubeClient client.Client, servicePath string, deploymentNamespace string) error {
func setupService(ctx context.Context, kubeClient client.Client, servicePath string, deploymentNamespace string) error {
existingService := &corev1.Service{}
//check if service has already been created
err := kubeClient.Get(context.Background(), types.NamespacedName{Name: "onepassword-connect", Namespace: deploymentNamespace}, existingService)
err := kubeClient.Get(ctx, types.NamespacedName{Name: "onepassword-connect", Namespace: deploymentNamespace}, existingService)
if err != nil {
if errors.IsNotFound(err) {
logConnectSetup.Info("No existing Connect service found. Creating Service")
return createService(kubeClient, servicePath, deploymentNamespace)
return createService(ctx, kubeClient, servicePath, deploymentNamespace)
}
}
return err
}
func createService(kubeClient client.Client, servicePath string, deploymentNamespace string) error {
func createService(ctx context.Context, kubeClient client.Client, servicePath string, deploymentNamespace string) error {
f, err := os.Open(servicePath)
if err != nil {
return err
@@ -108,7 +108,7 @@ func createService(kubeClient client.Client, servicePath string, deploymentNames
return err
}
err = kubeClient.Create(context.Background(), service)
err = kubeClient.Create(ctx, service)
if err != nil {
return err
}

View File

@@ -15,6 +15,7 @@ import (
var defaultNamespacedName = types.NamespacedName{Name: "onepassword-connect", Namespace: "default"}
func TestServiceSetup(t *testing.T) {
ctx := context.Background()
// Register operator types with the runtime scheme.
s := scheme.Scheme
@@ -25,7 +26,7 @@ func TestServiceSetup(t *testing.T) {
// Create a fake client to mock API calls.
client := fake.NewClientBuilder().WithScheme(s).WithRuntimeObjects(objs...).Build()
err := setupService(client, "../../config/connect/service.yaml", defaultNamespacedName.Namespace)
err := setupService(ctx, client, "../../config/connect/service.yaml", defaultNamespacedName.Namespace)
if err != nil {
t.Errorf("Error Setting Up Connect: %v", err)
@@ -33,13 +34,14 @@ func TestServiceSetup(t *testing.T) {
// check that service was created
service := &corev1.Service{}
err = client.Get(context.TODO(), defaultNamespacedName, service)
err = client.Get(ctx, defaultNamespacedName, service)
if err != nil {
t.Errorf("Error Setting Up Connect service: %v", err)
}
}
func TestDeploymentSetup(t *testing.T) {
ctx := context.Background()
// Register operator types with the runtime scheme.
s := scheme.Scheme
@@ -50,7 +52,7 @@ func TestDeploymentSetup(t *testing.T) {
// Create a fake client to mock API calls.
client := fake.NewClientBuilder().WithScheme(s).WithRuntimeObjects(objs...).Build()
err := setupDeployment(client, "../../config/connect/deployment.yaml", defaultNamespacedName.Namespace)
err := setupDeployment(ctx, client, "../../config/connect/deployment.yaml", defaultNamespacedName.Namespace)
if err != nil {
t.Errorf("Error Setting Up Connect: %v", err)
@@ -58,7 +60,7 @@ func TestDeploymentSetup(t *testing.T) {
// check that deployment was created
deployment := &appsv1.Deployment{}
err = client.Get(context.TODO(), defaultNamespacedName, deployment)
err = client.Get(ctx, defaultNamespacedName, deployment)
if err != nil {
t.Errorf("Error Setting Up Connect deployment: %v", err)
}

View File

@@ -1,42 +1,44 @@
package onepassword
import (
"context"
"fmt"
"strings"
"github.com/1Password/connect-sdk-go/connect"
"github.com/1Password/connect-sdk-go/onepassword"
logf "sigs.k8s.io/controller-runtime/pkg/log"
opclient "github.com/1Password/onepassword-operator/pkg/onepassword/client"
"github.com/1Password/onepassword-operator/pkg/onepassword/model"
)
var logger = logf.Log.WithName("retrieve_item")
func GetOnePasswordItemByPath(opConnectClient connect.Client, path string) (*onepassword.Item, error) {
vaultValue, itemValue, err := ParseVaultAndItemFromPath(path)
func GetOnePasswordItemByPath(ctx context.Context, opClient opclient.Client, path string) (*model.Item, error) {
vaultNameOrID, itemNameOrID, err := ParseVaultAndItemFromPath(path)
if err != nil {
return nil, err
}
vaultId, err := getVaultId(opConnectClient, vaultValue)
vaultID, err := getVaultID(ctx, opClient, vaultNameOrID)
if err != nil {
return nil, err
return nil, fmt.Errorf("failed to 'getVaultID' for vaultNameOrID='%s': %w", vaultNameOrID, err)
}
itemId, err := getItemId(opConnectClient, itemValue, vaultId)
itemID, err := getItemID(ctx, opClient, vaultID, itemNameOrID)
if err != nil {
return nil, err
return nil, fmt.Errorf("faild to 'getItemID' for vaultID='%s' and itemNameOrID='%s': %w", vaultID, itemNameOrID, err)
}
item, err := opConnectClient.GetItem(itemId, vaultId)
item, err := opClient.GetItemByID(ctx, vaultID, itemID)
if err != nil {
return nil, err
return nil, fmt.Errorf("faield to 'GetItemByID' for vaultID='%s' and itemID='%s': %w", vaultID, itemID, err)
}
for _, file := range item.Files {
_, err := opConnectClient.GetFileContent(file)
for i, file := range item.Files {
content, err := opClient.GetFileContent(ctx, vaultID, itemID, file.ID)
if err != nil {
return nil, err
}
item.Files[i].SetContent(content)
}
return item, nil
@@ -50,15 +52,15 @@ func ParseVaultAndItemFromPath(path string) (string, string, error) {
return "", "", fmt.Errorf("%q is not an acceptable path for One Password item. Must be of the format: `vaults/{vault_id}/items/{item_id}`", path)
}
func getVaultId(client connect.Client, vaultIdentifier string) (string, error) {
if !IsValidClientUUID(vaultIdentifier) {
vaults, err := client.GetVaultsByTitle(vaultIdentifier)
func getVaultID(ctx context.Context, client opclient.Client, vaultNameOrID string) (string, error) {
if !IsValidClientUUID(vaultNameOrID) {
vaults, err := client.GetVaultsByTitle(ctx, vaultNameOrID)
if err != nil {
return "", err
}
if len(vaults) == 0 {
return "", fmt.Errorf("No vaults found with identifier %q", vaultIdentifier)
return "", fmt.Errorf("No vaults found with identifier %q", vaultNameOrID)
}
oldestVault := vaults[0]
@@ -68,22 +70,22 @@ func getVaultId(client connect.Client, vaultIdentifier string) (string, error) {
oldestVault = returnedVault
}
}
logger.Info(fmt.Sprintf("%v 1Password vaults found with the title %q. Will use vault %q as it is the oldest.", len(vaults), vaultIdentifier, oldestVault.ID))
logger.Info(fmt.Sprintf("%v 1Password vaults found with the title %q. Will use vault %q as it is the oldest.", len(vaults), vaultNameOrID, oldestVault.ID))
}
vaultIdentifier = oldestVault.ID
vaultNameOrID = oldestVault.ID
}
return vaultIdentifier, nil
return vaultNameOrID, nil
}
func getItemId(client connect.Client, itemIdentifier string, vaultId string) (string, error) {
if !IsValidClientUUID(itemIdentifier) {
items, err := client.GetItemsByTitle(itemIdentifier, vaultId)
func getItemID(ctx context.Context, client opclient.Client, vaultId, itemNameOrID string) (string, error) {
if !IsValidClientUUID(itemNameOrID) {
items, err := client.GetItemsByTitle(ctx, vaultId, itemNameOrID)
if err != nil {
return "", err
}
if len(items) == 0 {
return "", fmt.Errorf("No items found with identifier %q", itemIdentifier)
return "", fmt.Errorf("No items found with identifier %q", itemNameOrID)
}
oldestItem := items[0]
@@ -93,9 +95,9 @@ func getItemId(client connect.Client, itemIdentifier string, vaultId string) (st
oldestItem = returnedItem
}
}
logger.Info(fmt.Sprintf("%v 1Password items found with the title %q. Will use item %q as it is the oldest.", len(items), itemIdentifier, oldestItem.ID))
logger.Info(fmt.Sprintf("%v 1Password items found with the title %q. Will use item %q as it is the oldest.", len(items), itemNameOrID, oldestItem.ID))
}
itemIdentifier = oldestItem.ID
itemNameOrID = oldestItem.ID
}
return itemIdentifier, nil
return itemNameOrID, nil
}

View File

@@ -0,0 +1,27 @@
package model
import (
"errors"
)
// File represents a file stored in 1Password.
type File struct {
ID string
Name string
Size int
ContentPath string
content []byte
}
// Content returns the content of the file if they have been loaded and returns an error if they have not been loaded.
// Use `client.GetFileContent(file *File)` instead to make sure the content is fetched automatically if not present.
func (f *File) Content() ([]byte, error) {
if f.content == nil {
return nil, errors.New("file content not loaded")
}
return f.content, nil
}
func (f *File) SetContent(content []byte) {
f.content = content
}

View File

@@ -0,0 +1,94 @@
package model
import (
"time"
connect "github.com/1Password/connect-sdk-go/onepassword"
sdk "github.com/1password/onepassword-sdk-go"
)
// Item represents 1Password item.
type Item struct {
ID string
VaultID string
Version int
Tags []string
Fields []ItemField
Files []File
CreatedAt time.Time
}
// FromConnectItem populates the Item from a Connect item.
func (i *Item) FromConnectItem(item *connect.Item) {
i.ID = item.ID
i.VaultID = item.Vault.ID
i.Version = item.Version
for _, tag := range item.Tags {
i.Tags = append(i.Tags, tag)
}
for _, field := range item.Fields {
i.Fields = append(i.Fields, ItemField{
Label: field.Label,
Value: field.Value,
})
}
for _, file := range item.Files {
i.Files = append(i.Files, File{
ID: file.ID,
Name: file.Name,
Size: file.Size,
})
}
i.CreatedAt = item.CreatedAt
}
// FromSDKItem populates the Item from an SDK item.
func (i *Item) FromSDKItem(item *sdk.Item) {
i.ID = item.ID
i.VaultID = item.VaultID
i.Version = int(item.Version)
i.Tags = make([]string, len(item.Tags))
copy(i.Tags, item.Tags)
for _, field := range item.Fields {
i.Fields = append(i.Fields, ItemField{
Label: field.Title,
Value: field.Value,
})
}
for _, file := range item.Files {
i.Files = append(i.Files, File{
ID: file.Attributes.ID,
Name: file.Attributes.Name,
Size: int(file.Attributes.Size),
})
}
// Items of 'Document' category keeps file information in the Document field.
if item.Category == sdk.ItemCategoryDocument {
i.Files = append(i.Files, File{
ID: item.Document.ID,
Name: item.Document.Name,
Size: int(item.Document.Size),
})
}
i.CreatedAt = item.CreatedAt
}
// FromSDKItemOverview populates the Item from an SDK item overview.
func (i *Item) FromSDKItemOverview(item *sdk.ItemOverview) {
i.ID = item.ID
i.VaultID = item.VaultID
i.Tags = make([]string, len(item.Tags))
copy(i.Tags, item.Tags)
i.CreatedAt = item.CreatedAt
}

View File

@@ -0,0 +1,7 @@
package model
// ItemField Representation of a single field on an Item
type ItemField struct {
Label string
Value string
}

View File

@@ -0,0 +1,108 @@
package model
import (
"testing"
"time"
"github.com/stretchr/testify/require"
connect "github.com/1Password/connect-sdk-go/onepassword"
sdk "github.com/1password/onepassword-sdk-go"
)
func TestItem_FromConnectItem(t *testing.T) {
connectItem := &connect.Item{
ID: "test-item-id",
Vault: connect.ItemVault{
ID: "test-vault-id",
},
Version: 1,
Tags: []string{"tag1", "tag2"},
Fields: []*connect.ItemField{
{Label: "field1", Value: "value1"},
{Label: "field2", Value: "value2"},
},
Files: []*connect.File{
{ID: "file1", Name: "file1.txt", Size: 1234},
{ID: "file2", Name: "file2.txt", Size: 1234},
},
CreatedAt: time.Now(),
}
item := &Item{}
item.FromConnectItem(connectItem)
require.Equal(t, connectItem.ID, item.ID)
require.Equal(t, connectItem.Vault.ID, item.VaultID)
require.Equal(t, connectItem.Version, item.Version)
require.ElementsMatch(t, connectItem.Tags, item.Tags)
for i, field := range connectItem.Fields {
require.Equal(t, field.Label, item.Fields[i].Label)
require.Equal(t, field.Value, item.Fields[i].Value)
}
for i, file := range connectItem.Files {
require.Equal(t, file.ID, item.Files[i].ID)
require.Equal(t, file.Name, item.Files[i].Name)
require.Equal(t, file.Size, item.Files[i].Size)
}
require.Equal(t, connectItem.CreatedAt, item.CreatedAt)
}
func TestItem_FromSDKItem(t *testing.T) {
sdkItem := &sdk.Item{
ID: "test-item-id",
VaultID: "test-vault-id",
Version: 1,
Tags: []string{"tag1", "tag2"},
Fields: []sdk.ItemField{
{ID: "1", Title: "field1", Value: "value1"},
{ID: "2", Title: "field2", Value: "value2"},
},
Files: []sdk.ItemFile{
{Attributes: sdk.FileAttributes{Name: "file1.txt", Size: 1234}, FieldID: "file1"},
{Attributes: sdk.FileAttributes{Name: "file2.txt", Size: 1234}, FieldID: "file2"},
},
CreatedAt: time.Now(),
}
item := &Item{}
item.FromSDKItem(sdkItem)
require.Equal(t, sdkItem.ID, item.ID)
require.Equal(t, sdkItem.VaultID, item.VaultID)
require.Equal(t, int(sdkItem.Version), item.Version)
require.ElementsMatch(t, sdkItem.Tags, item.Tags)
for i, field := range sdkItem.Fields {
require.Equal(t, field.Title, item.Fields[i].Label)
require.Equal(t, field.Value, item.Fields[i].Value)
}
for i, file := range sdkItem.Files {
require.Equal(t, file.Attributes.ID, item.Files[i].ID)
require.Equal(t, file.Attributes.Name, item.Files[i].Name)
require.Equal(t, int(file.Attributes.Size), item.Files[i].Size)
}
require.Equal(t, sdkItem.CreatedAt, item.CreatedAt)
}
func TestItem_FromSDKItemOverview(t *testing.T) {
sdkItemOverview := &sdk.ItemOverview{
ID: "test-item-id",
VaultID: "test-vault-id",
Tags: []string{"tag1", "tag2"},
CreatedAt: time.Now(),
}
item := &Item{}
item.FromSDKItemOverview(sdkItemOverview)
require.Equal(t, sdkItemOverview.ID, item.ID)
require.Equal(t, sdkItemOverview.VaultID, item.VaultID)
require.ElementsMatch(t, sdkItemOverview.Tags, item.Tags)
require.Equal(t, sdkItemOverview.CreatedAt, item.CreatedAt)
}

View File

@@ -0,0 +1,23 @@
package model
import (
"time"
connect "github.com/1Password/connect-sdk-go/onepassword"
sdk "github.com/1password/onepassword-sdk-go"
)
type Vault struct {
ID string
CreatedAt time.Time
}
func (v *Vault) FromConnectVault(vault *connect.Vault) {
v.ID = vault.ID
v.CreatedAt = vault.CreatedAt
}
func (v *Vault) FromSDKVault(vault *sdk.VaultOverview) {
v.ID = vault.ID
v.CreatedAt = vault.CreatedAt
}

View File

@@ -0,0 +1,37 @@
package model
import (
"testing"
"time"
"github.com/stretchr/testify/require"
connect "github.com/1Password/connect-sdk-go/onepassword"
sdk "github.com/1password/onepassword-sdk-go"
)
func TestVault_FromConnectVault(t *testing.T) {
connectVault := &connect.Vault{
ID: "test-id",
CreatedAt: time.Now(),
}
vault := &Vault{}
vault.FromConnectVault(connectVault)
require.Equal(t, connectVault.ID, vault.ID)
require.Equal(t, connectVault.CreatedAt, vault.CreatedAt)
}
func TestVault_FromSDKVault(t *testing.T) {
sdkVault := &sdk.VaultOverview{
ID: "test-id",
CreatedAt: time.Now(),
}
vault := &Vault{}
vault.FromSDKVault(sdkVault)
require.Equal(t, sdkVault.ID, vault.ID)
require.Equal(t, sdkVault.CreatedAt, vault.CreatedAt)
}

View File

@@ -8,10 +8,10 @@ import (
onepasswordv1 "github.com/1Password/onepassword-operator/api/v1"
kubeSecrets "github.com/1Password/onepassword-operator/pkg/kubernetessecrets"
"github.com/1Password/onepassword-operator/pkg/logs"
opclient "github.com/1Password/onepassword-operator/pkg/onepassword/client"
"github.com/1Password/onepassword-operator/pkg/onepassword/model"
"github.com/1Password/onepassword-operator/pkg/utils"
"github.com/1Password/connect-sdk-go/connect"
"github.com/1Password/connect-sdk-go/onepassword"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
"sigs.k8s.io/controller-runtime/pkg/client"
@@ -23,37 +23,37 @@ const lockTag = "operator.1password.io:ignore-secret"
var log = logf.Log.WithName("update_op_kubernetes_secrets_task")
func NewManager(kubernetesClient client.Client, opConnectClient connect.Client, shouldAutoRestartDeploymentsGlobal bool) *SecretUpdateHandler {
func NewManager(kubernetesClient client.Client, opClient opclient.Client, shouldAutoRestartDeploymentsGlobal bool) *SecretUpdateHandler {
return &SecretUpdateHandler{
client: kubernetesClient,
opConnectClient: opConnectClient,
opClient: opClient,
shouldAutoRestartDeploymentsGlobal: shouldAutoRestartDeploymentsGlobal,
}
}
type SecretUpdateHandler struct {
client client.Client
opConnectClient connect.Client
opClient opclient.Client
shouldAutoRestartDeploymentsGlobal bool
}
func (h *SecretUpdateHandler) UpdateKubernetesSecretsTask() error {
updatedKubernetesSecrets, err := h.updateKubernetesSecrets()
func (h *SecretUpdateHandler) UpdateKubernetesSecretsTask(ctx context.Context) error {
updatedKubernetesSecrets, err := h.updateKubernetesSecrets(ctx)
if err != nil {
return err
}
return h.restartDeploymentsWithUpdatedSecrets(updatedKubernetesSecrets)
return h.restartDeploymentsWithUpdatedSecrets(ctx, updatedKubernetesSecrets)
}
func (h *SecretUpdateHandler) restartDeploymentsWithUpdatedSecrets(updatedSecretsByNamespace map[string]map[string]*corev1.Secret) error {
func (h *SecretUpdateHandler) restartDeploymentsWithUpdatedSecrets(ctx context.Context, updatedSecretsByNamespace map[string]map[string]*corev1.Secret) error {
// No secrets to update. Exit
if len(updatedSecretsByNamespace) == 0 || updatedSecretsByNamespace == nil {
return nil
}
deployments := &appsv1.DeploymentList{}
err := h.client.List(context.Background(), deployments)
err := h.client.List(ctx, deployments)
if err != nil {
log.Error(err, "Failed to list kubernetes deployments")
return err
@@ -63,7 +63,7 @@ func (h *SecretUpdateHandler) restartDeploymentsWithUpdatedSecrets(updatedSecret
return nil
}
setForAutoRestartByNamespaceMap, err := h.getIsSetForAutoRestartByNamespaceMap()
setForAutoRestartByNamespaceMap, err := h.getIsSetForAutoRestartByNamespaceMap(ctx)
if err != nil {
return err
}
@@ -78,7 +78,7 @@ func (h *SecretUpdateHandler) restartDeploymentsWithUpdatedSecrets(updatedSecret
}
for _, secret := range updatedDeploymentSecrets {
if isSecretSetForAutoRestart(secret, deployment, setForAutoRestartByNamespaceMap) {
h.restartDeployment(deployment)
h.restartDeployment(ctx, deployment)
continue
}
}
@@ -89,21 +89,21 @@ func (h *SecretUpdateHandler) restartDeploymentsWithUpdatedSecrets(updatedSecret
return nil
}
func (h *SecretUpdateHandler) restartDeployment(deployment *appsv1.Deployment) {
func (h *SecretUpdateHandler) restartDeployment(ctx context.Context, deployment *appsv1.Deployment) {
log.Info(fmt.Sprintf("Deployment %q at namespace %q references an updated secret. Restarting", deployment.GetName(), deployment.Namespace))
if deployment.Spec.Template.Annotations == nil {
deployment.Spec.Template.Annotations = map[string]string{}
}
deployment.Spec.Template.Annotations[RestartAnnotation] = time.Now().String()
err := h.client.Update(context.Background(), deployment)
err := h.client.Update(ctx, deployment)
if err != nil {
log.Error(err, "Problem restarting deployment")
}
}
func (h *SecretUpdateHandler) updateKubernetesSecrets() (map[string]map[string]*corev1.Secret, error) {
func (h *SecretUpdateHandler) updateKubernetesSecrets(ctx context.Context) (map[string]map[string]*corev1.Secret, error) {
secrets := &corev1.SecretList{}
err := h.client.List(context.Background(), secrets)
err := h.client.List(ctx, secrets)
if err != nil {
log.Error(err, "Failed to list kubernetes secrets")
return nil, err
@@ -121,22 +121,22 @@ func (h *SecretUpdateHandler) updateKubernetesSecrets() (map[string]map[string]*
OnePasswordItemPath := h.getPathFromOnePasswordItem(secret)
item, err := GetOnePasswordItemByPath(h.opConnectClient, OnePasswordItemPath)
item, err := GetOnePasswordItemByPath(ctx, h.opClient, OnePasswordItemPath)
if err != nil {
log.Error(err, "failed to retrieve 1Password item at path \"%s\" for secret \"%s\"", secret.Annotations[ItemPathAnnotation], secret.Name)
log.Error(err, fmt.Sprintf("failed to retrieve 1Password item at path %s for secret %s", secret.Annotations[ItemPathAnnotation], secret.Name))
continue
}
itemVersion := fmt.Sprint(item.Version)
itemPathString := fmt.Sprintf("vaults/%v/items/%v", item.Vault.ID, item.ID)
itemPathString := fmt.Sprintf("vaults/%v/items/%v", item.VaultID, item.ID)
if currentVersion != itemVersion || secret.Annotations[ItemPathAnnotation] != itemPathString {
if isItemLockedForForcedRestarts(item) {
log.V(logs.DebugLevel).Info(fmt.Sprintf("Secret '%v' has been updated in 1Password but is set to be ignored. Updates to an ignored secret will not trigger an update to a kubernetes secret or a rolling restart.", secret.GetName()))
secret.Annotations[VersionAnnotation] = itemVersion
secret.Annotations[ItemPathAnnotation] = itemPathString
if err := h.client.Update(context.Background(), &secret); err != nil {
log.Error(err, "failed to update secret %s annotations to version %d: %s", secret.Name, itemVersion, err)
if err := h.client.Update(ctx, &secret); err != nil {
log.Error(err, fmt.Sprintf("failed to update secret %s annotations to version %s", secret.Name, itemVersion))
continue
}
continue
@@ -146,8 +146,8 @@ func (h *SecretUpdateHandler) updateKubernetesSecrets() (map[string]map[string]*
secret.Annotations[ItemPathAnnotation] = itemPathString
secret.Data = kubeSecrets.BuildKubernetesSecretData(item.Fields, item.Files)
log.V(logs.DebugLevel).Info(fmt.Sprintf("New secret path: %v and version: %v", secret.Annotations[ItemPathAnnotation], secret.Annotations[VersionAnnotation]))
if err := h.client.Update(context.Background(), &secret); err != nil {
log.Error(err, "failed to update secret %s to version %d: %s", secret.Name, itemVersion, err)
if err := h.client.Update(ctx, &secret); err != nil {
log.Error(err, fmt.Sprintf("failed to update secret %s to version %s", secret.Name, itemVersion))
continue
}
if updatedSecrets[secret.Namespace] == nil {
@@ -159,7 +159,7 @@ func (h *SecretUpdateHandler) updateKubernetesSecrets() (map[string]map[string]*
return updatedSecrets, nil
}
func isItemLockedForForcedRestarts(item *onepassword.Item) bool {
func isItemLockedForForcedRestarts(item *model.Item) bool {
tags := item.Tags
for i := 0; i < len(tags); i++ {
if tags[i] == lockTag {
@@ -177,9 +177,9 @@ func isUpdatedSecret(secretName string, updatedSecrets map[string]*corev1.Secret
return false
}
func (h *SecretUpdateHandler) getIsSetForAutoRestartByNamespaceMap() (map[string]bool, error) {
func (h *SecretUpdateHandler) getIsSetForAutoRestartByNamespaceMap(ctx context.Context) (map[string]bool, error) {
namespaces := &corev1.NamespaceList{}
err := h.client.List(context.Background(), namespaces)
err := h.client.List(ctx, namespaces)
if err != nil {
log.Error(err, "Failed to list kubernetes namespaces")
return nil, err
@@ -218,7 +218,7 @@ func isSecretSetForAutoRestart(secret *corev1.Secret, deployment *appsv1.Deploym
restartDeploymentBool, err := utils.StringToBool(restartDeployment)
if err != nil {
log.Error(err, "Error parsing %v annotation on Secret %v. Must be true or false. Defaulting to false.", RestartDeploymentsAnnotation, secret.Name)
log.Error(err, fmt.Sprintf("Error parsing %s annotation on Secret %s. Must be true or false. Defaulting to false.", RestartDeploymentsAnnotation, secret.Name))
return false
}
return restartDeploymentBool
@@ -233,7 +233,7 @@ func isDeploymentSetForAutoRestart(deployment *appsv1.Deployment, setForAutoRest
restartDeploymentBool, err := utils.StringToBool(restartDeployment)
if err != nil {
log.Error(err, "Error parsing %v annotation on Deployment %v. Must be true or false. Defaulting to false.", RestartDeploymentsAnnotation, deployment.Name)
log.Error(err, fmt.Sprintf("Error parsing %s annotation on Deployment %s. Must be true or false. Defaulting to false.", RestartDeploymentsAnnotation, deployment.Name))
return false
}
return restartDeploymentBool
@@ -248,7 +248,7 @@ func (h *SecretUpdateHandler) isNamespaceSetToAutoRestart(namespace *corev1.Name
restartDeploymentBool, err := utils.StringToBool(restartDeployment)
if err != nil {
log.Error(err, "Error parsing %v annotation on Namespace %v. Must be true or false. Defaulting to false.", RestartDeploymentsAnnotation, namespace.Name)
log.Error(err, fmt.Sprintf("Error parsing %s annotation on Namespace %s. Must be true or false. Defaulting to false.", RestartDeploymentsAnnotation, namespace.Name))
return false
}
return restartDeploymentBool

View File

@@ -4,11 +4,14 @@ import (
"context"
"fmt"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/1Password/onepassword-operator/pkg/mocks"
"github.com/1Password/onepassword-operator/pkg/onepassword/model"
"github.com/1Password/connect-sdk-go/onepassword"
"github.com/stretchr/testify/assert"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
errors2 "k8s.io/apimachinery/pkg/api/errors"
@@ -784,7 +787,7 @@ var tests = []testUpdateSecretTask{
func TestUpdateSecretHandler(t *testing.T) {
for _, testData := range tests {
t.Run(testData.testName, func(t *testing.T) {
ctx := context.Background()
// Register operator types with the runtime scheme.
s := scheme.Scheme
s.AddKnownTypes(appsv1.SchemeGroupVersion, testData.existingDeployment)
@@ -802,23 +805,15 @@ func TestUpdateSecretHandler(t *testing.T) {
// Create a fake client to mock API calls.
cl := fake.NewClientBuilder().WithScheme(s).WithRuntimeObjects(objs...).Build()
opConnectClient := &mocks.TestClient{}
mocks.DoGetItemFunc = func(uuid string, vaultUUID string) (*onepassword.Item, error) {
item := onepassword.Item{}
item.Fields = generateFields(testData.opItem["username"], testData.opItem["password"])
item.Version = itemVersion
item.Vault.ID = vaultUUID
item.ID = uuid
return &item, nil
}
mockOpClient := &mocks.TestClient{}
mockOpClient.On("GetItemByID", mock.Anything, mock.Anything).Return(createItem(), nil)
h := &SecretUpdateHandler{
client: cl,
opConnectClient: opConnectClient,
opClient: mockOpClient,
shouldAutoRestartDeploymentsGlobal: testData.globalAutoRestartEnabled,
}
err := h.UpdateKubernetesSecretsTask()
err := h.UpdateKubernetesSecretsTask(ctx)
assert.Equal(t, testData.expectedError, err)
@@ -831,7 +826,7 @@ func TestUpdateSecretHandler(t *testing.T) {
// Check if Secret has been created and has the correct data
secret := &corev1.Secret{}
err = cl.Get(context.TODO(), types.NamespacedName{Name: expectedSecretName, Namespace: namespace}, secret)
err = cl.Get(ctx, types.NamespacedName{Name: expectedSecretName, Namespace: namespace}, secret)
if testData.expectedResultSecret == nil {
assert.Error(t, err)
@@ -845,7 +840,7 @@ func TestUpdateSecretHandler(t *testing.T) {
//check if deployment has been restarted
deployment := &appsv1.Deployment{}
err = cl.Get(context.TODO(), types.NamespacedName{Name: testData.existingDeployment.Name, Namespace: namespace}, deployment)
err = cl.Get(ctx, types.NamespacedName{Name: testData.existingDeployment.Name, Namespace: namespace}, deployment)
_, ok := deployment.Spec.Template.Annotations[RestartAnnotation]
if ok {
@@ -879,16 +874,23 @@ func TestIsUpdatedSecret(t *testing.T) {
assert.True(t, isUpdatedSecret(secretName, updatedSecrets))
}
func generateFields(username, password string) []*onepassword.ItemField {
fields := []*onepassword.ItemField{
{
Label: "username",
Value: username,
},
{
Label: "password",
Value: password,
func createItem() *model.Item {
return &model.Item{
ID: itemId,
VaultID: vaultId,
Version: itemVersion,
Tags: []string{"tag1", "tag2"},
Fields: []model.ItemField{
{
Label: "username",
Value: username,
},
{
Label: "password",
Value: password,
},
},
Files: []model.File{},
CreatedAt: time.Now(),
}
return fields
}

View File

@@ -1,21 +0,0 @@
MIT License
Copyright (c) 2021 1Password
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@@ -1,869 +0,0 @@
package connect
import (
"bytes"
"encoding/json"
"fmt"
"io"
"io/ioutil"
"net/http"
"net/url"
"os"
"path/filepath"
"reflect"
"regexp"
"github.com/opentracing/opentracing-go"
"github.com/opentracing/opentracing-go/ext"
jaegerClientConfig "github.com/uber/jaeger-client-go/config"
"github.com/uber/jaeger-client-go/zipkin"
"github.com/1Password/connect-sdk-go/onepassword"
)
const (
defaultUserAgent = "connect-sdk-go/%s"
)
var (
vaultUUIDError = fmt.Errorf("malformed vault uuid provided")
itemUUIDError = fmt.Errorf("malformed item uuid provided")
fileUUIDError = fmt.Errorf("malformed file uuid provided")
)
// Client Represents an available 1Password Connect API to connect to
type Client interface {
GetVaults() ([]onepassword.Vault, error)
GetVault(uuid string) (*onepassword.Vault, error)
GetVaultByUUID(uuid string) (*onepassword.Vault, error)
GetVaultByTitle(title string) (*onepassword.Vault, error)
GetVaultsByTitle(uuid string) ([]onepassword.Vault, error)
GetItems(vaultQuery string) ([]onepassword.Item, error)
GetItem(itemQuery, vaultQuery string) (*onepassword.Item, error)
GetItemByUUID(uuid string, vaultQuery string) (*onepassword.Item, error)
GetItemByTitle(title string, vaultQuery string) (*onepassword.Item, error)
GetItemsByTitle(title string, vaultQuery string) ([]onepassword.Item, error)
CreateItem(item *onepassword.Item, vaultQuery string) (*onepassword.Item, error)
UpdateItem(item *onepassword.Item, vaultQuery string) (*onepassword.Item, error)
DeleteItem(item *onepassword.Item, vaultQuery string) error
DeleteItemByID(itemUUID string, vaultQuery string) error
DeleteItemByTitle(title string, vaultQuery string) error
GetFiles(itemQuery string, vaultQuery string) ([]onepassword.File, error)
GetFile(uuid string, itemQuery string, vaultQuery string) (*onepassword.File, error)
GetFileContent(file *onepassword.File) ([]byte, error)
DownloadFile(file *onepassword.File, targetDirectory string, overwrite bool) (string, error)
LoadStructFromItemByUUID(config interface{}, itemUUID string, vaultQuery string) error
LoadStructFromItemByTitle(config interface{}, itemTitle string, vaultQuery string) error
LoadStructFromItem(config interface{}, itemQuery string, vaultQuery string) error
LoadStruct(config interface{}) error
}
type httpClient interface {
Do(req *http.Request) (*http.Response, error)
}
const (
envHostVariable = "OP_CONNECT_HOST"
envTokenVariable = "OP_CONNECT_TOKEN"
)
// NewClientFromEnvironment Returns a Secret Service client assuming that your
// jwt is set in the OP_TOKEN environment variable
func NewClientFromEnvironment() (Client, error) {
host, found := os.LookupEnv(envHostVariable)
if !found {
return nil, fmt.Errorf("There is no hostname available in the %q variable", envHostVariable)
}
token, found := os.LookupEnv(envTokenVariable)
if !found {
return nil, fmt.Errorf("There is no token available in the %q variable", envTokenVariable)
}
return NewClient(host, token), nil
}
// NewClient Returns a Secret Service client for a given url and jwt
func NewClient(url string, token string) Client {
return NewClientWithUserAgent(url, token, fmt.Sprintf(defaultUserAgent, SDKVersion))
}
// NewClientWithUserAgent Returns a Secret Service client for a given url and jwt and identifies with userAgent
func NewClientWithUserAgent(url string, token string, userAgent string) Client {
if !opentracing.IsGlobalTracerRegistered() {
cfg := jaegerClientConfig.Configuration{}
zipkinPropagator := zipkin.NewZipkinB3HTTPHeaderPropagator()
cfg.InitGlobalTracer(
userAgent,
jaegerClientConfig.Injector(opentracing.HTTPHeaders, zipkinPropagator),
jaegerClientConfig.Extractor(opentracing.HTTPHeaders, zipkinPropagator),
jaegerClientConfig.ZipkinSharedRPCSpan(true),
)
}
return &restClient{
URL: url,
Token: token,
userAgent: userAgent,
tracer: opentracing.GlobalTracer(),
client: http.DefaultClient,
}
}
type restClient struct {
URL string
Token string
userAgent string
tracer opentracing.Tracer
client httpClient
}
// GetVaults Get a list of all available vaults
func (rs *restClient) GetVaults() ([]onepassword.Vault, error) {
span := rs.tracer.StartSpan("GetVaults")
defer span.Finish()
vaultURL := fmt.Sprintf("/v1/vaults")
request, err := rs.buildRequest(http.MethodGet, vaultURL, http.NoBody, span)
if err != nil {
return nil, err
}
response, err := rs.client.Do(request)
if err != nil {
return nil, err
}
var vaults []onepassword.Vault
if err := parseResponse(response, http.StatusOK, &vaults); err != nil {
return nil, err
}
return vaults, nil
}
// GetVault Get a vault based on its name or ID
func (rs *restClient) GetVault(vaultQuery string) (*onepassword.Vault, error) {
span := rs.tracer.StartSpan("GetVault")
defer span.Finish()
if vaultQuery == "" {
return nil, fmt.Errorf("Please provide either the vault name or its ID.")
}
if !isValidUUID(vaultQuery) {
return rs.GetVaultByTitle(vaultQuery)
}
return rs.GetVaultByUUID(vaultQuery)
}
func (rs *restClient) GetVaultByUUID(uuid string) (*onepassword.Vault, error) {
if !isValidUUID(uuid) {
return nil, vaultUUIDError
}
span := rs.tracer.StartSpan("GetVaultByUUID")
defer span.Finish()
vaultURL := fmt.Sprintf("/v1/vaults/%s", uuid)
request, err := rs.buildRequest(http.MethodGet, vaultURL, http.NoBody, span)
if err != nil {
return nil, err
}
response, err := rs.client.Do(request)
if err != nil {
return nil, err
}
var vault onepassword.Vault
if err := parseResponse(response, http.StatusOK, &vault); err != nil {
return nil, err
}
return &vault, nil
}
func (rs *restClient) GetVaultByTitle(vaultName string) (*onepassword.Vault, error) {
span := rs.tracer.StartSpan("GetVaultByTitle")
defer span.Finish()
vaults, err := rs.GetVaultsByTitle(vaultName)
if err != nil {
return nil, err
}
if len(vaults) != 1 {
return nil, fmt.Errorf("Found %d vaults with title %q", len(vaults), vaultName)
}
return &vaults[0], nil
}
func (rs *restClient) GetVaultsByTitle(title string) ([]onepassword.Vault, error) {
span := rs.tracer.StartSpan("GetVaultsByTitle")
defer span.Finish()
filter := url.QueryEscape(fmt.Sprintf("title eq \"%s\"", title))
itemURL := fmt.Sprintf("/v1/vaults?filter=%s", filter)
request, err := rs.buildRequest(http.MethodGet, itemURL, http.NoBody, span)
if err != nil {
return nil, err
}
response, err := rs.client.Do(request)
if err != nil {
return nil, err
}
var vaults []onepassword.Vault
if err := parseResponse(response, http.StatusOK, &vaults); err != nil {
return nil, err
}
return vaults, nil
}
func (rs *restClient) getVaultUUID(vaultQuery string) (string, error) {
if vaultQuery == "" {
return "", fmt.Errorf("Please provide either the vault name or its ID.")
}
if isValidUUID(vaultQuery) {
return vaultQuery, nil
}
vault, err := rs.GetVaultByTitle(vaultQuery)
if err != nil {
return "", err
}
return vault.ID, nil
}
// GetItem Get a specific Item from the 1Password Connect API by either title or UUID
func (rs *restClient) GetItem(itemQuery string, vaultQuery string) (*onepassword.Item, error) {
span := rs.tracer.StartSpan("GetItem")
defer span.Finish()
if itemQuery == "" {
return nil, fmt.Errorf("Please provide either the item name or its ID.")
}
if isValidUUID(itemQuery) {
item, err := rs.GetItemByUUID(itemQuery, vaultQuery)
if item != nil {
return item, err
}
}
return rs.GetItemByTitle(itemQuery, vaultQuery)
}
// GetItemByUUID Get a specific Item from the 1Password Connect API by its UUID
func (rs *restClient) GetItemByUUID(uuid string, vaultQuery string) (*onepassword.Item, error) {
if !isValidUUID(uuid) {
return nil, itemUUIDError
}
vaultUUID, err := rs.getVaultUUID(vaultQuery)
if err != nil {
return nil, err
}
span := rs.tracer.StartSpan("GetItemByUUID")
defer span.Finish()
itemURL := fmt.Sprintf("/v1/vaults/%s/items/%s", vaultUUID, uuid)
request, err := rs.buildRequest(http.MethodGet, itemURL, http.NoBody, span)
if err != nil {
return nil, err
}
response, err := rs.client.Do(request)
if err != nil {
return nil, err
}
var item onepassword.Item
if err := parseResponse(response, http.StatusOK, &item); err != nil {
return nil, err
}
return &item, nil
}
func (rs *restClient) GetItemByTitle(title string, vaultQuery string) (*onepassword.Item, error) {
vaultUUID, err := rs.getVaultUUID(vaultQuery)
if err != nil {
return nil, err
}
span := rs.tracer.StartSpan("GetItemByTitle")
defer span.Finish()
items, err := rs.GetItemsByTitle(title, vaultUUID)
if err != nil {
return nil, err
}
if len(items) != 1 {
return nil, fmt.Errorf("Found %d item(s) in vault %q with title %q", len(items), vaultUUID, title)
}
return &items[0], nil
}
func (rs *restClient) GetItemsByTitle(title string, vaultQuery string) ([]onepassword.Item, error) {
vaultUUID, err := rs.getVaultUUID(vaultQuery)
if err != nil {
return nil, err
}
span := rs.tracer.StartSpan("GetItemsByTitle")
defer span.Finish()
filter := url.QueryEscape(fmt.Sprintf("title eq \"%s\"", title))
itemURL := fmt.Sprintf("/v1/vaults/%s/items?filter=%s", vaultUUID, filter)
request, err := rs.buildRequest(http.MethodGet, itemURL, http.NoBody, span)
if err != nil {
return nil, err
}
response, err := rs.client.Do(request)
if err != nil {
return nil, err
}
var itemSummaries []onepassword.Item
if err := parseResponse(response, http.StatusOK, &itemSummaries); err != nil {
return nil, err
}
items := make([]onepassword.Item, len(itemSummaries))
for i, itemSummary := range itemSummaries {
tempItem, err := rs.GetItem(itemSummary.ID, itemSummary.Vault.ID)
if err != nil {
return nil, err
}
items[i] = *tempItem
}
return items, nil
}
func (rs *restClient) GetItems(vaultQuery string) ([]onepassword.Item, error) {
vaultUUID, err := rs.getVaultUUID(vaultQuery)
if err != nil {
return nil, err
}
span := rs.tracer.StartSpan("GetItems")
defer span.Finish()
itemURL := fmt.Sprintf("/v1/vaults/%s/items", vaultUUID)
request, err := rs.buildRequest(http.MethodGet, itemURL, http.NoBody, span)
if err != nil {
return nil, err
}
response, err := rs.client.Do(request)
if err != nil {
return nil, err
}
var items []onepassword.Item
if err := parseResponse(response, http.StatusOK, &items); err != nil {
return nil, err
}
return items, nil
}
func (rs *restClient) getItemUUID(itemQuery, vaultQuery string) (string, error) {
if itemQuery == "" {
return "", fmt.Errorf("Please provide either the item name or its ID.")
}
if isValidUUID(itemQuery) {
return itemQuery, nil
}
item, err := rs.GetItemByTitle(itemQuery, vaultQuery)
if err != nil {
return "", err
}
return item.ID, nil
}
// CreateItem Create a new item in a specified vault
func (rs *restClient) CreateItem(item *onepassword.Item, vaultQuery string) (*onepassword.Item, error) {
vaultUUID, err := rs.getVaultUUID(vaultQuery)
if err != nil {
return nil, err
}
span := rs.tracer.StartSpan("CreateItem")
defer span.Finish()
itemURL := fmt.Sprintf("/v1/vaults/%s/items", vaultUUID)
itemBody, err := json.Marshal(item)
if err != nil {
return nil, err
}
request, err := rs.buildRequest(http.MethodPost, itemURL, bytes.NewBuffer(itemBody), span)
if err != nil {
return nil, err
}
response, err := rs.client.Do(request)
if err != nil {
return nil, err
}
var newItem onepassword.Item
if err := parseResponse(response, http.StatusOK, &newItem); err != nil {
return nil, err
}
return &newItem, nil
}
// UpdateItem Update a new item in a specified vault
func (rs *restClient) UpdateItem(item *onepassword.Item, vaultUUID string) (*onepassword.Item, error) {
span := rs.tracer.StartSpan("UpdateItem")
defer span.Finish()
itemURL := fmt.Sprintf("/v1/vaults/%s/items/%s", item.Vault.ID, item.ID)
itemBody, err := json.Marshal(item)
if err != nil {
return nil, err
}
request, err := rs.buildRequest(http.MethodPut, itemURL, bytes.NewBuffer(itemBody), span)
if err != nil {
return nil, err
}
response, err := rs.client.Do(request)
if err != nil {
return nil, err
}
var newItem onepassword.Item
if err := parseResponse(response, http.StatusOK, &newItem); err != nil {
return nil, err
}
return &newItem, nil
}
// DeleteItem Delete a new item in a specified vault
func (rs *restClient) DeleteItem(item *onepassword.Item, vaultUUID string) error {
span := rs.tracer.StartSpan("DeleteItem")
defer span.Finish()
itemURL := fmt.Sprintf("/v1/vaults/%s/items/%s", item.Vault.ID, item.ID)
request, err := rs.buildRequest(http.MethodDelete, itemURL, http.NoBody, span)
if err != nil {
return err
}
response, err := rs.client.Do(request)
if err != nil {
return err
}
if err := parseResponse(response, http.StatusNoContent, nil); err != nil {
return err
}
return nil
}
// DeleteItemByID Delete a new item in a specified vault, specifying the item's uuid
func (rs *restClient) DeleteItemByID(itemUUID string, vaultQuery string) error {
if !isValidUUID(itemUUID) {
return itemUUIDError
}
vaultUUID, err := rs.getVaultUUID(vaultQuery)
if err != nil {
return err
}
span := rs.tracer.StartSpan("DeleteItemByID")
defer span.Finish()
itemURL := fmt.Sprintf("/v1/vaults/%s/items/%s", vaultUUID, itemUUID)
request, err := rs.buildRequest(http.MethodDelete, itemURL, http.NoBody, span)
if err != nil {
return err
}
response, err := rs.client.Do(request)
if err != nil {
return err
}
if err := parseResponse(response, http.StatusNoContent, nil); err != nil {
return err
}
return nil
}
// DeleteItemByTitle Delete a new item in a specified vault, specifying the item's title
func (rs *restClient) DeleteItemByTitle(title string, vaultQuery string) error {
span := rs.tracer.StartSpan("DeleteItemByTitle")
defer span.Finish()
item, err := rs.GetItemByTitle(title, vaultQuery)
if err != nil {
return err
}
return rs.DeleteItem(item, item.Vault.ID)
}
func (rs *restClient) GetFiles(itemQuery string, vaultQuery string) ([]onepassword.File, error) {
vaultUUID, err := rs.getVaultUUID(vaultQuery)
if err != nil {
return nil, err
}
itemUUID, err := rs.getItemUUID(itemQuery, vaultQuery)
if err != nil {
return nil, err
}
span := rs.tracer.StartSpan("GetFiles")
defer span.Finish()
jsonURL := fmt.Sprintf("/v1/vaults/%s/items/%s/files", vaultUUID, itemUUID)
request, err := rs.buildRequest(http.MethodGet, jsonURL, http.NoBody, span)
if err != nil {
return nil, err
}
response, err := rs.client.Do(request)
if err != nil {
return nil, err
}
if err := expectMinimumConnectVersion(response, version{1, 3, 0}); err != nil {
return nil, err
}
var files []onepassword.File
if err := parseResponse(response, http.StatusOK, &files); err != nil {
return nil, err
}
return files, nil
}
// GetFile Get a specific File in a specified item.
// This does not include the file contents. Call GetFileContent() to load the file's content.
func (rs *restClient) GetFile(uuid string, itemQuery string, vaultQuery string) (*onepassword.File, error) {
if !isValidUUID(uuid) {
return nil, fileUUIDError
}
vaultUUID, err := rs.getVaultUUID(vaultQuery)
if err != nil {
return nil, err
}
itemUUID, err := rs.getItemUUID(itemQuery, vaultQuery)
if err != nil {
return nil, err
}
span := rs.tracer.StartSpan("GetFile")
defer span.Finish()
itemURL := fmt.Sprintf("/v1/vaults/%s/items/%s/files/%s", vaultUUID, itemUUID, uuid)
request, err := rs.buildRequest(http.MethodGet, itemURL, http.NoBody, span)
if err != nil {
return nil, err
}
response, err := rs.client.Do(request)
if err != nil {
return nil, err
}
if err := expectMinimumConnectVersion(response, version{1, 3, 0}); err != nil {
return nil, err
}
var file onepassword.File
if err := parseResponse(response, http.StatusOK, &file); err != nil {
return nil, err
}
return &file, nil
}
// GetFileContent retrieves the file's content.
// If the file's content have previously been fetched, those contents are returned without making another request.
func (rs *restClient) GetFileContent(file *onepassword.File) ([]byte, error) {
if content, err := file.Content(); err == nil {
return content, nil
}
response, err := rs.retrieveDocumentContent(file)
if err != nil {
return nil, err
}
content, err := readResponseBody(response, http.StatusOK)
if err != nil {
return nil, err
}
file.SetContent(content)
return content, nil
}
func (rs *restClient) DownloadFile(file *onepassword.File, targetDirectory string, overwriteIfExists bool) (string, error) {
response, err := rs.retrieveDocumentContent(file)
if err != nil {
return "", err
}
path := filepath.Join(targetDirectory, filepath.Base(file.Name))
var osFile *os.File
if overwriteIfExists {
osFile, err = createFile(path)
if err != nil {
return "", err
}
} else {
_, err = os.Stat(path)
if os.IsNotExist(err) {
osFile, err = createFile(path)
if err != nil {
return "", err
}
} else {
return "", fmt.Errorf("a file already exists under the %s path. In order to overwrite it, set `overwriteIfExists` to true", path)
}
}
defer osFile.Close()
if _, err = io.Copy(osFile, response.Body); err != nil {
return "", err
}
return path, nil
}
func (rs *restClient) retrieveDocumentContent(file *onepassword.File) (*http.Response, error) {
span := rs.tracer.StartSpan("GetFileContent")
defer span.Finish()
request, err := rs.buildRequest(http.MethodGet, file.ContentPath, http.NoBody, span)
if err != nil {
return nil, err
}
response, err := rs.client.Do(request)
if err != nil {
return nil, err
}
if err := expectMinimumConnectVersion(response, version{1, 3, 0}); err != nil {
return nil, err
}
return response, nil
}
func createFile(path string) (*os.File, error) {
osFile, err := os.Create(path)
if err != nil {
return nil, err
}
err = os.Chmod(path, 0600)
if err != nil {
return nil, err
}
return osFile, nil
}
func (rs *restClient) buildRequest(method string, path string, body io.Reader, span opentracing.Span) (*http.Request, error) {
url := fmt.Sprintf("%s%s", rs.URL, path)
request, err := http.NewRequest(method, url, body)
if err != nil {
return nil, err
}
request.Header.Set("Content-Type", "application/json")
request.Header.Set("Authorization", fmt.Sprintf("Bearer %s", rs.Token))
request.Header.Set("User-Agent", rs.userAgent)
ext.SpanKindRPCClient.Set(span)
ext.HTTPUrl.Set(span, path)
ext.HTTPMethod.Set(span, method)
rs.tracer.Inject(span.Context(), opentracing.HTTPHeaders, opentracing.HTTPHeadersCarrier(request.Header))
return request, nil
}
func loadToStruct(item *parsedItem, config reflect.Value) error {
t := config.Type()
for i := 0; i < t.NumField(); i++ {
value := config.Field(i)
field := t.Field(i)
if !value.CanSet() {
return fmt.Errorf("cannot load config into private fields")
}
item.fields = append(item.fields, &field)
item.values = append(item.values, &value)
}
return nil
}
// LoadStructFromItem Load configuration values based on struct tag from one 1P item.
// It accepts as parameters item title/UUID and vault title/UUID.
func (rs *restClient) LoadStructFromItem(i interface{}, itemQuery string, vaultQuery string) error {
if itemQuery == "" {
return fmt.Errorf("Please provide either the item name or its ID.")
}
if isValidUUID(itemQuery) {
return rs.LoadStructFromItemByUUID(i, itemQuery, vaultQuery)
}
return rs.LoadStructFromItemByTitle(i, itemQuery, vaultQuery)
}
// LoadStructFromItemByUUID Load configuration values based on struct tag from one 1P item.
func (rs *restClient) LoadStructFromItemByUUID(i interface{}, itemUUID string, vaultQuery string) error {
vaultUUID, err := rs.getVaultUUID(vaultQuery)
if err != nil {
return err
}
if !isValidUUID(itemUUID) {
return itemUUIDError
}
config, err := checkStruct(i)
if err != nil {
return err
}
item := parsedItem{}
item.itemUUID = itemUUID
item.vaultUUID = vaultUUID
if err := loadToStruct(&item, config); err != nil {
return err
}
if err := setValuesForTag(rs, &item, false); err != nil {
return err
}
return nil
}
// LoadStructFromItemByTitle Load configuration values based on struct tag from one 1P item
func (rs *restClient) LoadStructFromItemByTitle(i interface{}, itemTitle string, vaultQuery string) error {
vaultUUID, err := rs.getVaultUUID(vaultQuery)
if err != nil {
return err
}
config, err := checkStruct(i)
if err != nil {
return err
}
item := parsedItem{}
item.itemTitle = itemTitle
item.vaultUUID = vaultUUID
if err := loadToStruct(&item, config); err != nil {
return err
}
if err := setValuesForTag(rs, &item, true); err != nil {
return err
}
return nil
}
// LoadStruct Load configuration values based on struct tag
func (rs *restClient) LoadStruct(i interface{}) error {
config, err := checkStruct(i)
if err != nil {
return err
}
t := config.Type()
// Multiple fields may be from a single item so we will collect them
items := map[string]parsedItem{}
// Fetch the Vault from the environment
vaultUUID, envVarFound := os.LookupEnv(envVaultVar)
for i := 0; i < t.NumField(); i++ {
value := config.Field(i)
field := t.Field(i)
tag := field.Tag.Get(itemTag)
if tag == "" {
continue
}
if !value.CanSet() {
return fmt.Errorf("Cannot load config into private fields")
}
itemVault, err := vaultUUIDForField(&field, vaultUUID, envVarFound)
if err != nil {
return err
}
if !isValidUUID(itemVault) {
return vaultUUIDError
}
key := fmt.Sprintf("%s/%s", itemVault, tag)
parsed := items[key]
parsed.vaultUUID = itemVault
parsed.itemTitle = tag
parsed.fields = append(parsed.fields, &field)
parsed.values = append(parsed.values, &value)
items[key] = parsed
}
for _, item := range items {
if err := setValuesForTag(rs, &item, true); err != nil {
return err
}
}
return nil
}
func parseResponse(resp *http.Response, expectedStatusCode int, result interface{}) error {
body, err := readResponseBody(resp, expectedStatusCode)
if err != nil {
return err
}
if result != nil {
if err := json.Unmarshal(body, result); err != nil {
return fmt.Errorf("decoding response: %s", err)
}
}
return nil
}
func readResponseBody(resp *http.Response, expectedStatusCode int) ([]byte, error) {
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
if resp.StatusCode != expectedStatusCode {
var errResp onepassword.Error
if json.Valid(body) {
if err := json.Unmarshal(body, &errResp); err != nil {
return nil, fmt.Errorf("decoding error response: %s", err)
}
} else {
errResp.StatusCode = resp.StatusCode
errResp.Message = http.StatusText(resp.StatusCode)
}
return nil, &errResp
}
return body, nil
}
func isValidUUID(u string) bool {
r := regexp.MustCompile("^[a-z0-9]{26}$")
return r.MatchString(u)
}

View File

@@ -1,209 +0,0 @@
package connect
import (
"fmt"
"reflect"
"strconv"
"strings"
"github.com/1Password/connect-sdk-go/onepassword"
)
const (
vaultTag = "opvault"
itemTag = "opitem"
sectionTag = "opsection"
fieldTag = "opfield"
urlTag = "opurl"
envVaultVar = "OP_VAULT"
)
type parsedItem struct {
vaultUUID string
itemUUID string
itemTitle string
fields []*reflect.StructField
values []*reflect.Value
}
func checkStruct(i interface{}) (reflect.Value, error) {
configP := reflect.ValueOf(i)
if configP.Kind() != reflect.Ptr {
return reflect.Value{}, fmt.Errorf("you must pass a pointer to Config struct")
}
config := configP.Elem()
if config.Kind() != reflect.Struct {
return reflect.Value{}, fmt.Errorf("config values can only be loaded into a struct")
}
return config, nil
}
func vaultUUIDForField(field *reflect.StructField, vaultUUID string, envVaultFound bool) (string, error) {
// Check to see if a specific vault has been specified on the field
// If the env vault id has not been found and item doesn't have a vault
// return an error
if vaultUUIDTag := field.Tag.Get(vaultTag); vaultUUIDTag == "" {
if !envVaultFound {
return "", fmt.Errorf("There is no vault for %q field", field.Name)
}
} else {
return vaultUUIDTag, nil
}
return vaultUUID, nil
}
func setValuesForTag(client Client, parsedItem *parsedItem, byTitle bool) error {
var item *onepassword.Item
var err error
if byTitle {
item, err = client.GetItemByTitle(parsedItem.itemTitle, parsedItem.vaultUUID)
} else {
item, err = client.GetItem(parsedItem.itemUUID, parsedItem.vaultUUID)
}
if err != nil {
return err
}
for i, field := range parsedItem.fields {
value := parsedItem.values[i]
if field.Type == reflect.TypeOf(onepassword.ItemURL{}) {
url := &onepassword.ItemURL{
Primary: urlPrimaryForName(field.Tag.Get(urlTag), item.URLs),
Label: urlLabelForName(field.Tag.Get(urlTag), item.URLs),
URL: urlURLForName(field.Tag.Get(urlTag), item.URLs),
}
value.Set(reflect.ValueOf(*url))
continue
}
path := fmt.Sprintf("%s.%s", field.Tag.Get(sectionTag), field.Tag.Get(fieldTag))
if path == "." {
if field.Type == reflect.TypeOf(onepassword.Item{}) {
value.Set(reflect.ValueOf(*item))
continue
}
return fmt.Errorf("There is no %q specified for %q", fieldTag, field.Name)
}
if strings.HasSuffix(path, ".") {
if field.Type == reflect.TypeOf(onepassword.ItemSection{}) {
section := &onepassword.ItemSection{
ID: sectionIDForName(field.Tag.Get(sectionTag), item.Sections),
Label: sectionLabelForName(field.Tag.Get(sectionTag), item.Sections),
}
value.Set(reflect.ValueOf(*section))
continue
}
}
sectionID := sectionIDForName(field.Tag.Get(sectionTag), item.Sections)
for _, f := range item.Fields {
fieldSectionID := ""
if f.Section != nil {
fieldSectionID = f.Section.ID
}
if fieldSectionID == sectionID && f.Label == field.Tag.Get(fieldTag) {
if err := setValue(value, f.Value); err != nil {
return err
}
break
}
}
}
return nil
}
func setValue(value *reflect.Value, toSet string) error {
switch value.Kind() {
case reflect.String:
value.SetString(toSet)
case reflect.Int:
v, err := strconv.Atoi(toSet)
if err != nil {
return err
}
value.SetInt(int64(v))
default:
return fmt.Errorf("Unsupported type %q. Only string, int64, and onepassword.Item are supported", value.Kind())
}
return nil
}
func sectionIDForName(name string, sections []*onepassword.ItemSection) string {
if sections == nil {
return ""
}
for _, s := range sections {
if name == strings.ToLower(s.Label) {
return s.ID
}
}
return ""
}
func sectionLabelForName(name string, sections []*onepassword.ItemSection) string {
if sections == nil {
return ""
}
for _, s := range sections {
if name == strings.ToLower(s.Label) {
return s.Label
}
}
return ""
}
func urlPrimaryForName(name string, itemURLs []onepassword.ItemURL) bool {
if itemURLs == nil {
return false
}
for _, url := range itemURLs {
if url.Label == strings.ToLower(name) {
return url.Primary
}
}
return false
}
func urlLabelForName(name string, itemURLs []onepassword.ItemURL) string {
if itemURLs == nil {
return ""
}
for _, url := range itemURLs {
if url.Label == strings.ToLower(name) {
return url.Label
}
}
return ""
}
func urlURLForName(name string, itemURLs []onepassword.ItemURL) string {
if itemURLs == nil {
return ""
}
for _, url := range itemURLs {
if url.Label == strings.ToLower(name) {
return url.URL
}
}
return ""
}

View File

@@ -1,104 +0,0 @@
package connect
import (
"errors"
"fmt"
"net/http"
"strconv"
"strings"
)
// SDKVersion is the latest Semantic Version of the library
// Do not rename this variable without changing the regex in the Makefile
const SDKVersion = "1.5.3"
const VersionHeaderKey = "1Password-Connect-Version"
// expectMinimumConnectVersion returns an error if the provided minimum version for Connect is lower than the version
// reported in the response from Connect.
func expectMinimumConnectVersion(resp *http.Response, minimumVersion version) error {
serverVersion, err := getServerVersion(resp)
if err != nil {
// Return gracefully if server version cannot be determined reliably
return nil
}
if !serverVersion.IsGreaterOrEqualThan(minimumVersion) {
return fmt.Errorf("need at least version %s of Connect for this function, detected version %s. Please update your Connect server", minimumVersion, serverVersion)
}
return nil
}
func getServerVersion(resp *http.Response) (serverVersion, error) {
versionHeader := resp.Header.Get(VersionHeaderKey)
if versionHeader == "" {
// The last version without the version header was v1.2.0
return serverVersion{
version: version{1, 2, 0},
orEarlier: true,
}, nil
}
return parseServerVersion(versionHeader)
}
type version struct {
major int
minor int
patch int
}
// serverVersion describes the version reported by the server.
type serverVersion struct {
version
// orEarlier is true if the version is derived from the lack of a version header from the server.
orEarlier bool
}
func (v version) String() string {
return fmt.Sprintf("%d.%d.%d", v.major, v.minor, v.patch)
}
func (v serverVersion) String() string {
if v.orEarlier {
return v.version.String() + " (or earlier)"
}
return v.version.String()
}
// IsGreaterOrEqualThan returns true if the lefthand-side version is equal to or or a higher version than the provided
// minimum according to the semantic versioning rules.
func (v version) IsGreaterOrEqualThan(min version) bool {
if v.major != min.major {
// Different major version
return v.major > min.major
}
if v.minor != min.minor {
// Same major, but different minor version
return v.minor > min.minor
}
// Same major and minor version
return v.patch >= min.patch
}
func parseServerVersion(v string) (serverVersion, error) {
spl := strings.Split(v, ".")
if len(spl) != 3 {
return serverVersion{}, errors.New("wrong length")
}
var res [3]int
for i := range res {
tmp, err := strconv.Atoi(spl[i])
if err != nil {
return serverVersion{}, err
}
res[i] = tmp
}
return serverVersion{
version: version{
major: res[0],
minor: res[1],
patch: res[2],
},
}, nil
}

View File

@@ -1,21 +0,0 @@
package onepassword
import "fmt"
// Error is an error returned by the Connect API.
type Error struct {
StatusCode int `json:"status"`
Message string `json:"message"`
}
func (e *Error) Error() string {
return fmt.Sprintf("status %d: %s", e.StatusCode, e.Message)
}
func (e *Error) Is(target error) bool {
t, ok := target.(*Error)
if !ok {
return false
}
return t.Message == e.Message && t.StatusCode == e.StatusCode
}

View File

@@ -1,49 +0,0 @@
package onepassword
import (
"encoding/json"
"errors"
)
type File struct {
ID string `json:"id"`
Name string `json:"name"`
Section *ItemSection `json:"section,omitempty"`
Size int `json:"size"`
ContentPath string `json:"content_path"`
content []byte
}
func (f *File) UnmarshalJSON(data []byte) error {
var jsonFile struct {
ID string `json:"id"`
Name string `json:"name"`
Section *ItemSection `json:"section,omitempty"`
Size int `json:"size"`
ContentPath string `json:"content_path"`
Content []byte `json:"content,omitempty"`
}
if err := json.Unmarshal(data, &jsonFile); err != nil {
return err
}
f.ID = jsonFile.ID
f.Name = jsonFile.Name
f.Section = jsonFile.Section
f.Size = jsonFile.Size
f.ContentPath = jsonFile.ContentPath
f.content = jsonFile.Content
return nil
}
// Content returns the content of the file if they have been loaded and returns an error if they have not been loaded.
// Use `client.GetFileContent(file *File)` instead to make sure the content is fetched automatically if not present.
func (f *File) Content() ([]byte, error) {
if f.content == nil {
return nil, errors.New("file content not loaded")
}
return f.content, nil
}
func (f *File) SetContent(content []byte) {
f.content = content
}

View File

@@ -1,193 +0,0 @@
package onepassword
import (
"encoding/json"
"strings"
"time"
)
// ItemCategory Represents the template of the Item
type ItemCategory string
type ItemFieldPurpose string
type ItemFieldType string
const (
Login ItemCategory = "LOGIN"
Password ItemCategory = "PASSWORD"
ApiCredential ItemCategory = "API_CREDENTIAL"
Server ItemCategory = "SERVER"
Database ItemCategory = "DATABASE"
CreditCard ItemCategory = "CREDIT_CARD"
Membership ItemCategory = "MEMBERSHIP"
Passport ItemCategory = "PASSPORT"
SoftwareLicense ItemCategory = "SOFTWARE_LICENSE"
OutdoorLicense ItemCategory = "OUTDOOR_LICENSE"
SecureNote ItemCategory = "SECURE_NOTE"
WirelessRouter ItemCategory = "WIRELESS_ROUTER"
BankAccount ItemCategory = "BANK_ACCOUNT"
DriverLicense ItemCategory = "DRIVER_LICENSE"
Identity ItemCategory = "IDENTITY"
RewardProgram ItemCategory = "REWARD_PROGRAM"
Document ItemCategory = "DOCUMENT"
EmailAccount ItemCategory = "EMAIL_ACCOUNT"
SocialSecurityNumber ItemCategory = "SOCIAL_SECURITY_NUMBER"
MedicalRecord ItemCategory = "MEDICAL_RECORD"
SSHKey ItemCategory = "SSH_KEY"
Custom ItemCategory = "CUSTOM"
FieldPurposeUsername ItemFieldPurpose = "USERNAME"
FieldPurposePassword ItemFieldPurpose = "PASSWORD"
FieldPurposeNotes ItemFieldPurpose = "NOTES"
FieldTypeAddress ItemFieldType = "ADDRESS"
FieldTypeConcealed ItemFieldType = "CONCEALED"
FieldTypeCreditCardNumber ItemFieldType = "CREDIT_CARD_NUMBER"
FieldTypeCreditCardType ItemFieldType = "CREDIT_CARD_TYPE"
FieldTypeDate ItemFieldType = "DATE"
FieldTypeEmail ItemFieldType = "EMAIL"
FieldTypeGender ItemFieldType = "GENDER"
FieldTypeMenu ItemFieldType = "MENU"
FieldTypeMonthYear ItemFieldType = "MONTH_YEAR"
FieldTypeOTP ItemFieldType = "OTP"
FieldTypePhone ItemFieldType = "PHONE"
FieldTypeReference ItemFieldType = "REFERENCE"
FieldTypeString ItemFieldType = "STRING"
FieldTypeURL ItemFieldType = "URL"
FieldTypeFile ItemFieldType = "FILE"
FieldTypeSSHKey ItemFieldType = "SSH_KEY"
FieldTypeUnknown ItemFieldType = "UNKNOWN"
)
// UnmarshalJSON Unmarshall Item Category enum strings to Go string enums
func (ic *ItemCategory) UnmarshalJSON(b []byte) error {
var s string
json.Unmarshal(b, &s)
category := ItemCategory(s)
switch category {
case Login, Password, Server, Database, CreditCard, Membership, Passport, SoftwareLicense,
OutdoorLicense, SecureNote, WirelessRouter, BankAccount, DriverLicense, Identity, RewardProgram,
Document, EmailAccount, SocialSecurityNumber, ApiCredential, MedicalRecord, SSHKey:
*ic = category
default:
*ic = Custom
}
return nil
}
// Item represents an item returned to the consumer
type Item struct {
ID string `json:"id"`
Title string `json:"title"`
URLs []ItemURL `json:"urls,omitempty"`
Favorite bool `json:"favorite,omitempty"`
Tags []string `json:"tags,omitempty"`
Version int `json:"version,omitempty"`
Vault ItemVault `json:"vault"`
Category ItemCategory `json:"category,omitempty"` // TODO: switch this to `category`
Sections []*ItemSection `json:"sections,omitempty"`
Fields []*ItemField `json:"fields,omitempty"`
Files []*File `json:"files,omitempty"`
LastEditedBy string `json:"lastEditedBy,omitempty"`
CreatedAt time.Time `json:"createdAt,omitempty"`
UpdatedAt time.Time `json:"updatedAt,omitempty"`
// Deprecated: Connect does not return trashed items.
Trashed bool `json:"trashed,omitempty"`
}
// ItemVault represents the Vault the Item is found in
type ItemVault struct {
ID string `json:"id"`
}
// ItemURL is a simplified item URL
type ItemURL struct {
Primary bool `json:"primary,omitempty"`
Label string `json:"label,omitempty"`
URL string `json:"href"`
}
// ItemSection Representation of a Section on an item
type ItemSection struct {
ID string `json:"id,omitempty"`
Label string `json:"label,omitempty"`
}
// GeneratorRecipe Representation of a "recipe" used to generate a field
type GeneratorRecipe struct {
Length int `json:"length,omitempty"`
CharacterSets []string `json:"characterSets,omitempty"`
ExcludeCharacters string `json:"excludeCharacters,omitempty"`
}
// ItemField Representation of a single field on an Item
type ItemField struct {
ID string `json:"id"`
Section *ItemSection `json:"section,omitempty"`
Type ItemFieldType `json:"type"`
Purpose ItemFieldPurpose `json:"purpose,omitempty"`
Label string `json:"label,omitempty"`
Value string `json:"value,omitempty"`
Generate bool `json:"generate,omitempty"`
Recipe *GeneratorRecipe `json:"recipe,omitempty"`
Entropy float64 `json:"entropy,omitempty"`
TOTP string `json:"totp,omitempty"`
}
// GetValue Retrieve the value of a field on the item by its label. To specify a
// field from a specific section pass in <section label>.<field label>. If
// no field matching the selector is found return "".
func (i *Item) GetValue(field string) string {
if i == nil || len(i.Fields) == 0 {
return ""
}
sectionFilter := false
sectionLabel := ""
fieldLabel := field
if strings.Contains(field, ".") {
parts := strings.Split(field, ".")
// Test to make sure the . isn't the last character
if len(parts) == 2 {
sectionFilter = true
sectionLabel = parts[0]
fieldLabel = parts[1]
}
}
for _, f := range i.Fields {
if sectionFilter {
if f.Section != nil {
if sectionLabel != i.SectionLabelForID(f.Section.ID) {
continue
}
}
}
if fieldLabel == f.Label {
return f.Value
}
}
return ""
}
func (i *Item) SectionLabelForID(id string) string {
if i != nil || len(i.Sections) > 0 {
for _, s := range i.Sections {
if s.ID == id {
return s.Label
}
}
}
return ""
}

View File

@@ -1,46 +0,0 @@
package onepassword
import (
"encoding/json"
"time"
)
// Vault represents a 1password Vault
type Vault struct {
ID string `json:"id"`
Name string `json:"name,omitempty"`
Description string `json:"description,omitempty"`
AttrVersion int `json:"attributeVersion,omitempty"`
ContentVersion int `json:"contentVersion,omitempty"`
Items int `json:"items,omitempty"`
Type VaultType `json:"type,omitempty"`
CreatedAt time.Time `json:"createdAt,omitempty"`
UpdatedAt time.Time `json:"updatedAt,omitempty"`
}
// VaultType Representation of what the Vault Type is
type VaultType string
const (
PersonalVault VaultType = "PERSONAL"
EveryoneVault VaultType = "EVERYONE"
TransferVault VaultType = "TRANSFER"
UserCreatedVault VaultType = "USER_CREATED"
UnknownVault VaultType = "UNKNOWN"
)
// UnmarshalJSON Unmarshall Vault Type enum strings to Go string enums
func (vt *VaultType) UnmarshalJSON(b []byte) error {
var s string
json.Unmarshal(b, &s)
vaultType := VaultType(s)
switch vaultType {
case PersonalVault, EveryoneVault, TransferVault, UserCreatedVault:
*vt = vaultType
default:
*vt = UnknownVault
}
return nil
}

View File

@@ -1,20 +0,0 @@
Copyright (C) 2013 Blake Mizerany
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

File diff suppressed because it is too large Load Diff

View File

@@ -1,316 +0,0 @@
// Package quantile computes approximate quantiles over an unbounded data
// stream within low memory and CPU bounds.
//
// A small amount of accuracy is traded to achieve the above properties.
//
// Multiple streams can be merged before calling Query to generate a single set
// of results. This is meaningful when the streams represent the same type of
// data. See Merge and Samples.
//
// For more detailed information about the algorithm used, see:
//
// Effective Computation of Biased Quantiles over Data Streams
//
// http://www.cs.rutgers.edu/~muthu/bquant.pdf
package quantile
import (
"math"
"sort"
)
// Sample holds an observed value and meta information for compression. JSON
// tags have been added for convenience.
type Sample struct {
Value float64 `json:",string"`
Width float64 `json:",string"`
Delta float64 `json:",string"`
}
// Samples represents a slice of samples. It implements sort.Interface.
type Samples []Sample
func (a Samples) Len() int { return len(a) }
func (a Samples) Less(i, j int) bool { return a[i].Value < a[j].Value }
func (a Samples) Swap(i, j int) { a[i], a[j] = a[j], a[i] }
type invariant func(s *stream, r float64) float64
// NewLowBiased returns an initialized Stream for low-biased quantiles
// (e.g. 0.01, 0.1, 0.5) where the needed quantiles are not known a priori, but
// error guarantees can still be given even for the lower ranks of the data
// distribution.
//
// The provided epsilon is a relative error, i.e. the true quantile of a value
// returned by a query is guaranteed to be within (1±Epsilon)*Quantile.
//
// See http://www.cs.rutgers.edu/~muthu/bquant.pdf for time, space, and error
// properties.
func NewLowBiased(epsilon float64) *Stream {
ƒ := func(s *stream, r float64) float64 {
return 2 * epsilon * r
}
return newStream(ƒ)
}
// NewHighBiased returns an initialized Stream for high-biased quantiles
// (e.g. 0.01, 0.1, 0.5) where the needed quantiles are not known a priori, but
// error guarantees can still be given even for the higher ranks of the data
// distribution.
//
// The provided epsilon is a relative error, i.e. the true quantile of a value
// returned by a query is guaranteed to be within 1-(1±Epsilon)*(1-Quantile).
//
// See http://www.cs.rutgers.edu/~muthu/bquant.pdf for time, space, and error
// properties.
func NewHighBiased(epsilon float64) *Stream {
ƒ := func(s *stream, r float64) float64 {
return 2 * epsilon * (s.n - r)
}
return newStream(ƒ)
}
// NewTargeted returns an initialized Stream concerned with a particular set of
// quantile values that are supplied a priori. Knowing these a priori reduces
// space and computation time. The targets map maps the desired quantiles to
// their absolute errors, i.e. the true quantile of a value returned by a query
// is guaranteed to be within (Quantile±Epsilon).
//
// See http://www.cs.rutgers.edu/~muthu/bquant.pdf for time, space, and error properties.
func NewTargeted(targetMap map[float64]float64) *Stream {
// Convert map to slice to avoid slow iterations on a map.
// ƒ is called on the hot path, so converting the map to a slice
// beforehand results in significant CPU savings.
targets := targetMapToSlice(targetMap)
ƒ := func(s *stream, r float64) float64 {
var m = math.MaxFloat64
var f float64
for _, t := range targets {
if t.quantile*s.n <= r {
f = (2 * t.epsilon * r) / t.quantile
} else {
f = (2 * t.epsilon * (s.n - r)) / (1 - t.quantile)
}
if f < m {
m = f
}
}
return m
}
return newStream(ƒ)
}
type target struct {
quantile float64
epsilon float64
}
func targetMapToSlice(targetMap map[float64]float64) []target {
targets := make([]target, 0, len(targetMap))
for quantile, epsilon := range targetMap {
t := target{
quantile: quantile,
epsilon: epsilon,
}
targets = append(targets, t)
}
return targets
}
// Stream computes quantiles for a stream of float64s. It is not thread-safe by
// design. Take care when using across multiple goroutines.
type Stream struct {
*stream
b Samples
sorted bool
}
func newStream(ƒ invariant) *Stream {
x := &stream{ƒ: ƒ}
return &Stream{x, make(Samples, 0, 500), true}
}
// Insert inserts v into the stream.
func (s *Stream) Insert(v float64) {
s.insert(Sample{Value: v, Width: 1})
}
func (s *Stream) insert(sample Sample) {
s.b = append(s.b, sample)
s.sorted = false
if len(s.b) == cap(s.b) {
s.flush()
}
}
// Query returns the computed qth percentiles value. If s was created with
// NewTargeted, and q is not in the set of quantiles provided a priori, Query
// will return an unspecified result.
func (s *Stream) Query(q float64) float64 {
if !s.flushed() {
// Fast path when there hasn't been enough data for a flush;
// this also yields better accuracy for small sets of data.
l := len(s.b)
if l == 0 {
return 0
}
i := int(math.Ceil(float64(l) * q))
if i > 0 {
i -= 1
}
s.maybeSort()
return s.b[i].Value
}
s.flush()
return s.stream.query(q)
}
// Merge merges samples into the underlying streams samples. This is handy when
// merging multiple streams from separate threads, database shards, etc.
//
// ATTENTION: This method is broken and does not yield correct results. The
// underlying algorithm is not capable of merging streams correctly.
func (s *Stream) Merge(samples Samples) {
sort.Sort(samples)
s.stream.merge(samples)
}
// Reset reinitializes and clears the list reusing the samples buffer memory.
func (s *Stream) Reset() {
s.stream.reset()
s.b = s.b[:0]
}
// Samples returns stream samples held by s.
func (s *Stream) Samples() Samples {
if !s.flushed() {
return s.b
}
s.flush()
return s.stream.samples()
}
// Count returns the total number of samples observed in the stream
// since initialization.
func (s *Stream) Count() int {
return len(s.b) + s.stream.count()
}
func (s *Stream) flush() {
s.maybeSort()
s.stream.merge(s.b)
s.b = s.b[:0]
}
func (s *Stream) maybeSort() {
if !s.sorted {
s.sorted = true
sort.Sort(s.b)
}
}
func (s *Stream) flushed() bool {
return len(s.stream.l) > 0
}
type stream struct {
n float64
l []Sample
ƒ invariant
}
func (s *stream) reset() {
s.l = s.l[:0]
s.n = 0
}
func (s *stream) insert(v float64) {
s.merge(Samples{{v, 1, 0}})
}
func (s *stream) merge(samples Samples) {
// TODO(beorn7): This tries to merge not only individual samples, but
// whole summaries. The paper doesn't mention merging summaries at
// all. Unittests show that the merging is inaccurate. Find out how to
// do merges properly.
var r float64
i := 0
for _, sample := range samples {
for ; i < len(s.l); i++ {
c := s.l[i]
if c.Value > sample.Value {
// Insert at position i.
s.l = append(s.l, Sample{})
copy(s.l[i+1:], s.l[i:])
s.l[i] = Sample{
sample.Value,
sample.Width,
math.Max(sample.Delta, math.Floor(s.ƒ(s, r))-1),
// TODO(beorn7): How to calculate delta correctly?
}
i++
goto inserted
}
r += c.Width
}
s.l = append(s.l, Sample{sample.Value, sample.Width, 0})
i++
inserted:
s.n += sample.Width
r += sample.Width
}
s.compress()
}
func (s *stream) count() int {
return int(s.n)
}
func (s *stream) query(q float64) float64 {
t := math.Ceil(q * s.n)
t += math.Ceil(s.ƒ(s, t) / 2)
p := s.l[0]
var r float64
for _, c := range s.l[1:] {
r += p.Width
if r+c.Width+c.Delta > t {
return p.Value
}
p = c
}
return p.Value
}
func (s *stream) compress() {
if len(s.l) < 2 {
return
}
x := s.l[len(s.l)-1]
xi := len(s.l) - 1
r := s.n - 1 - x.Width
for i := len(s.l) - 2; i >= 0; i-- {
c := s.l[i]
if c.Width+x.Width+x.Delta <= s.ƒ(s, r) {
x.Width += c.Width
s.l[xi] = x
// Remove element at i.
copy(s.l[i:], s.l[i+1:])
s.l = s.l[:len(s.l)-1]
xi -= 1
} else {
x = c
xi = i
}
r -= c.Width
}
}
func (s *stream) samples() Samples {
samples := make(Samples, len(s.l))
copy(samples, s.l)
return samples
}

View File

@@ -1,22 +0,0 @@
Copyright (c) 2016 Caleb Spare
MIT License
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

View File

@@ -1,72 +0,0 @@
# xxhash
[![Go Reference](https://pkg.go.dev/badge/github.com/cespare/xxhash/v2.svg)](https://pkg.go.dev/github.com/cespare/xxhash/v2)
[![Test](https://github.com/cespare/xxhash/actions/workflows/test.yml/badge.svg)](https://github.com/cespare/xxhash/actions/workflows/test.yml)
xxhash is a Go implementation of the 64-bit [xxHash] algorithm, XXH64. This is a
high-quality hashing algorithm that is much faster than anything in the Go
standard library.
This package provides a straightforward API:
```
func Sum64(b []byte) uint64
func Sum64String(s string) uint64
type Digest struct{ ... }
func New() *Digest
```
The `Digest` type implements hash.Hash64. Its key methods are:
```
func (*Digest) Write([]byte) (int, error)
func (*Digest) WriteString(string) (int, error)
func (*Digest) Sum64() uint64
```
The package is written with optimized pure Go and also contains even faster
assembly implementations for amd64 and arm64. If desired, the `purego` build tag
opts into using the Go code even on those architectures.
[xxHash]: http://cyan4973.github.io/xxHash/
## Compatibility
This package is in a module and the latest code is in version 2 of the module.
You need a version of Go with at least "minimal module compatibility" to use
github.com/cespare/xxhash/v2:
* 1.9.7+ for Go 1.9
* 1.10.3+ for Go 1.10
* Go 1.11 or later
I recommend using the latest release of Go.
## Benchmarks
Here are some quick benchmarks comparing the pure-Go and assembly
implementations of Sum64.
| input size | purego | asm |
| ---------- | --------- | --------- |
| 4 B | 1.3 GB/s | 1.2 GB/s |
| 16 B | 2.9 GB/s | 3.5 GB/s |
| 100 B | 6.9 GB/s | 8.1 GB/s |
| 4 KB | 11.7 GB/s | 16.7 GB/s |
| 10 MB | 12.0 GB/s | 17.3 GB/s |
These numbers were generated on Ubuntu 20.04 with an Intel Xeon Platinum 8252C
CPU using the following commands under Go 1.19.2:
```
benchstat <(go test -tags purego -benchtime 500ms -count 15 -bench 'Sum64$')
benchstat <(go test -benchtime 500ms -count 15 -bench 'Sum64$')
```
## Projects using this package
- [InfluxDB](https://github.com/influxdata/influxdb)
- [Prometheus](https://github.com/prometheus/prometheus)
- [VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics)
- [FreeCache](https://github.com/coocood/freecache)
- [FastCache](https://github.com/VictoriaMetrics/fastcache)

View File

@@ -1,10 +0,0 @@
#!/bin/bash
set -eu -o pipefail
# Small convenience script for running the tests with various combinations of
# arch/tags. This assumes we're running on amd64 and have qemu available.
go test ./...
go test -tags purego ./...
GOARCH=arm64 go test
GOARCH=arm64 go test -tags purego

View File

@@ -1,228 +0,0 @@
// Package xxhash implements the 64-bit variant of xxHash (XXH64) as described
// at http://cyan4973.github.io/xxHash/.
package xxhash
import (
"encoding/binary"
"errors"
"math/bits"
)
const (
prime1 uint64 = 11400714785074694791
prime2 uint64 = 14029467366897019727
prime3 uint64 = 1609587929392839161
prime4 uint64 = 9650029242287828579
prime5 uint64 = 2870177450012600261
)
// Store the primes in an array as well.
//
// The consts are used when possible in Go code to avoid MOVs but we need a
// contiguous array of the assembly code.
var primes = [...]uint64{prime1, prime2, prime3, prime4, prime5}
// Digest implements hash.Hash64.
type Digest struct {
v1 uint64
v2 uint64
v3 uint64
v4 uint64
total uint64
mem [32]byte
n int // how much of mem is used
}
// New creates a new Digest that computes the 64-bit xxHash algorithm.
func New() *Digest {
var d Digest
d.Reset()
return &d
}
// Reset clears the Digest's state so that it can be reused.
func (d *Digest) Reset() {
d.v1 = primes[0] + prime2
d.v2 = prime2
d.v3 = 0
d.v4 = -primes[0]
d.total = 0
d.n = 0
}
// Size always returns 8 bytes.
func (d *Digest) Size() int { return 8 }
// BlockSize always returns 32 bytes.
func (d *Digest) BlockSize() int { return 32 }
// Write adds more data to d. It always returns len(b), nil.
func (d *Digest) Write(b []byte) (n int, err error) {
n = len(b)
d.total += uint64(n)
memleft := d.mem[d.n&(len(d.mem)-1):]
if d.n+n < 32 {
// This new data doesn't even fill the current block.
copy(memleft, b)
d.n += n
return
}
if d.n > 0 {
// Finish off the partial block.
c := copy(memleft, b)
d.v1 = round(d.v1, u64(d.mem[0:8]))
d.v2 = round(d.v2, u64(d.mem[8:16]))
d.v3 = round(d.v3, u64(d.mem[16:24]))
d.v4 = round(d.v4, u64(d.mem[24:32]))
b = b[c:]
d.n = 0
}
if len(b) >= 32 {
// One or more full blocks left.
nw := writeBlocks(d, b)
b = b[nw:]
}
// Store any remaining partial block.
copy(d.mem[:], b)
d.n = len(b)
return
}
// Sum appends the current hash to b and returns the resulting slice.
func (d *Digest) Sum(b []byte) []byte {
s := d.Sum64()
return append(
b,
byte(s>>56),
byte(s>>48),
byte(s>>40),
byte(s>>32),
byte(s>>24),
byte(s>>16),
byte(s>>8),
byte(s),
)
}
// Sum64 returns the current hash.
func (d *Digest) Sum64() uint64 {
var h uint64
if d.total >= 32 {
v1, v2, v3, v4 := d.v1, d.v2, d.v3, d.v4
h = rol1(v1) + rol7(v2) + rol12(v3) + rol18(v4)
h = mergeRound(h, v1)
h = mergeRound(h, v2)
h = mergeRound(h, v3)
h = mergeRound(h, v4)
} else {
h = d.v3 + prime5
}
h += d.total
b := d.mem[:d.n&(len(d.mem)-1)]
for ; len(b) >= 8; b = b[8:] {
k1 := round(0, u64(b[:8]))
h ^= k1
h = rol27(h)*prime1 + prime4
}
if len(b) >= 4 {
h ^= uint64(u32(b[:4])) * prime1
h = rol23(h)*prime2 + prime3
b = b[4:]
}
for ; len(b) > 0; b = b[1:] {
h ^= uint64(b[0]) * prime5
h = rol11(h) * prime1
}
h ^= h >> 33
h *= prime2
h ^= h >> 29
h *= prime3
h ^= h >> 32
return h
}
const (
magic = "xxh\x06"
marshaledSize = len(magic) + 8*5 + 32
)
// MarshalBinary implements the encoding.BinaryMarshaler interface.
func (d *Digest) MarshalBinary() ([]byte, error) {
b := make([]byte, 0, marshaledSize)
b = append(b, magic...)
b = appendUint64(b, d.v1)
b = appendUint64(b, d.v2)
b = appendUint64(b, d.v3)
b = appendUint64(b, d.v4)
b = appendUint64(b, d.total)
b = append(b, d.mem[:d.n]...)
b = b[:len(b)+len(d.mem)-d.n]
return b, nil
}
// UnmarshalBinary implements the encoding.BinaryUnmarshaler interface.
func (d *Digest) UnmarshalBinary(b []byte) error {
if len(b) < len(magic) || string(b[:len(magic)]) != magic {
return errors.New("xxhash: invalid hash state identifier")
}
if len(b) != marshaledSize {
return errors.New("xxhash: invalid hash state size")
}
b = b[len(magic):]
b, d.v1 = consumeUint64(b)
b, d.v2 = consumeUint64(b)
b, d.v3 = consumeUint64(b)
b, d.v4 = consumeUint64(b)
b, d.total = consumeUint64(b)
copy(d.mem[:], b)
d.n = int(d.total % uint64(len(d.mem)))
return nil
}
func appendUint64(b []byte, x uint64) []byte {
var a [8]byte
binary.LittleEndian.PutUint64(a[:], x)
return append(b, a[:]...)
}
func consumeUint64(b []byte) ([]byte, uint64) {
x := u64(b)
return b[8:], x
}
func u64(b []byte) uint64 { return binary.LittleEndian.Uint64(b) }
func u32(b []byte) uint32 { return binary.LittleEndian.Uint32(b) }
func round(acc, input uint64) uint64 {
acc += input * prime2
acc = rol31(acc)
acc *= prime1
return acc
}
func mergeRound(acc, val uint64) uint64 {
val = round(0, val)
acc ^= val
acc = acc*prime1 + prime4
return acc
}
func rol1(x uint64) uint64 { return bits.RotateLeft64(x, 1) }
func rol7(x uint64) uint64 { return bits.RotateLeft64(x, 7) }
func rol11(x uint64) uint64 { return bits.RotateLeft64(x, 11) }
func rol12(x uint64) uint64 { return bits.RotateLeft64(x, 12) }
func rol18(x uint64) uint64 { return bits.RotateLeft64(x, 18) }
func rol23(x uint64) uint64 { return bits.RotateLeft64(x, 23) }
func rol27(x uint64) uint64 { return bits.RotateLeft64(x, 27) }
func rol31(x uint64) uint64 { return bits.RotateLeft64(x, 31) }

View File

@@ -1,209 +0,0 @@
//go:build !appengine && gc && !purego
// +build !appengine
// +build gc
// +build !purego
#include "textflag.h"
// Registers:
#define h AX
#define d AX
#define p SI // pointer to advance through b
#define n DX
#define end BX // loop end
#define v1 R8
#define v2 R9
#define v3 R10
#define v4 R11
#define x R12
#define prime1 R13
#define prime2 R14
#define prime4 DI
#define round(acc, x) \
IMULQ prime2, x \
ADDQ x, acc \
ROLQ $31, acc \
IMULQ prime1, acc
// round0 performs the operation x = round(0, x).
#define round0(x) \
IMULQ prime2, x \
ROLQ $31, x \
IMULQ prime1, x
// mergeRound applies a merge round on the two registers acc and x.
// It assumes that prime1, prime2, and prime4 have been loaded.
#define mergeRound(acc, x) \
round0(x) \
XORQ x, acc \
IMULQ prime1, acc \
ADDQ prime4, acc
// blockLoop processes as many 32-byte blocks as possible,
// updating v1, v2, v3, and v4. It assumes that there is at least one block
// to process.
#define blockLoop() \
loop: \
MOVQ +0(p), x \
round(v1, x) \
MOVQ +8(p), x \
round(v2, x) \
MOVQ +16(p), x \
round(v3, x) \
MOVQ +24(p), x \
round(v4, x) \
ADDQ $32, p \
CMPQ p, end \
JLE loop
// func Sum64(b []byte) uint64
TEXT ·Sum64(SB), NOSPLIT|NOFRAME, $0-32
// Load fixed primes.
MOVQ ·primes+0(SB), prime1
MOVQ ·primes+8(SB), prime2
MOVQ ·primes+24(SB), prime4
// Load slice.
MOVQ b_base+0(FP), p
MOVQ b_len+8(FP), n
LEAQ (p)(n*1), end
// The first loop limit will be len(b)-32.
SUBQ $32, end
// Check whether we have at least one block.
CMPQ n, $32
JLT noBlocks
// Set up initial state (v1, v2, v3, v4).
MOVQ prime1, v1
ADDQ prime2, v1
MOVQ prime2, v2
XORQ v3, v3
XORQ v4, v4
SUBQ prime1, v4
blockLoop()
MOVQ v1, h
ROLQ $1, h
MOVQ v2, x
ROLQ $7, x
ADDQ x, h
MOVQ v3, x
ROLQ $12, x
ADDQ x, h
MOVQ v4, x
ROLQ $18, x
ADDQ x, h
mergeRound(h, v1)
mergeRound(h, v2)
mergeRound(h, v3)
mergeRound(h, v4)
JMP afterBlocks
noBlocks:
MOVQ ·primes+32(SB), h
afterBlocks:
ADDQ n, h
ADDQ $24, end
CMPQ p, end
JG try4
loop8:
MOVQ (p), x
ADDQ $8, p
round0(x)
XORQ x, h
ROLQ $27, h
IMULQ prime1, h
ADDQ prime4, h
CMPQ p, end
JLE loop8
try4:
ADDQ $4, end
CMPQ p, end
JG try1
MOVL (p), x
ADDQ $4, p
IMULQ prime1, x
XORQ x, h
ROLQ $23, h
IMULQ prime2, h
ADDQ ·primes+16(SB), h
try1:
ADDQ $4, end
CMPQ p, end
JGE finalize
loop1:
MOVBQZX (p), x
ADDQ $1, p
IMULQ ·primes+32(SB), x
XORQ x, h
ROLQ $11, h
IMULQ prime1, h
CMPQ p, end
JL loop1
finalize:
MOVQ h, x
SHRQ $33, x
XORQ x, h
IMULQ prime2, h
MOVQ h, x
SHRQ $29, x
XORQ x, h
IMULQ ·primes+16(SB), h
MOVQ h, x
SHRQ $32, x
XORQ x, h
MOVQ h, ret+24(FP)
RET
// func writeBlocks(d *Digest, b []byte) int
TEXT ·writeBlocks(SB), NOSPLIT|NOFRAME, $0-40
// Load fixed primes needed for round.
MOVQ ·primes+0(SB), prime1
MOVQ ·primes+8(SB), prime2
// Load slice.
MOVQ b_base+8(FP), p
MOVQ b_len+16(FP), n
LEAQ (p)(n*1), end
SUBQ $32, end
// Load vN from d.
MOVQ s+0(FP), d
MOVQ 0(d), v1
MOVQ 8(d), v2
MOVQ 16(d), v3
MOVQ 24(d), v4
// We don't need to check the loop condition here; this function is
// always called with at least one block of data to process.
blockLoop()
// Copy vN back to d.
MOVQ v1, 0(d)
MOVQ v2, 8(d)
MOVQ v3, 16(d)
MOVQ v4, 24(d)
// The number of bytes written is p minus the old base pointer.
SUBQ b_base+8(FP), p
MOVQ p, ret+32(FP)
RET

View File

@@ -1,183 +0,0 @@
//go:build !appengine && gc && !purego
// +build !appengine
// +build gc
// +build !purego
#include "textflag.h"
// Registers:
#define digest R1
#define h R2 // return value
#define p R3 // input pointer
#define n R4 // input length
#define nblocks R5 // n / 32
#define prime1 R7
#define prime2 R8
#define prime3 R9
#define prime4 R10
#define prime5 R11
#define v1 R12
#define v2 R13
#define v3 R14
#define v4 R15
#define x1 R20
#define x2 R21
#define x3 R22
#define x4 R23
#define round(acc, x) \
MADD prime2, acc, x, acc \
ROR $64-31, acc \
MUL prime1, acc
// round0 performs the operation x = round(0, x).
#define round0(x) \
MUL prime2, x \
ROR $64-31, x \
MUL prime1, x
#define mergeRound(acc, x) \
round0(x) \
EOR x, acc \
MADD acc, prime4, prime1, acc
// blockLoop processes as many 32-byte blocks as possible,
// updating v1, v2, v3, and v4. It assumes that n >= 32.
#define blockLoop() \
LSR $5, n, nblocks \
PCALIGN $16 \
loop: \
LDP.P 16(p), (x1, x2) \
LDP.P 16(p), (x3, x4) \
round(v1, x1) \
round(v2, x2) \
round(v3, x3) \
round(v4, x4) \
SUB $1, nblocks \
CBNZ nblocks, loop
// func Sum64(b []byte) uint64
TEXT ·Sum64(SB), NOSPLIT|NOFRAME, $0-32
LDP b_base+0(FP), (p, n)
LDP ·primes+0(SB), (prime1, prime2)
LDP ·primes+16(SB), (prime3, prime4)
MOVD ·primes+32(SB), prime5
CMP $32, n
CSEL LT, prime5, ZR, h // if n < 32 { h = prime5 } else { h = 0 }
BLT afterLoop
ADD prime1, prime2, v1
MOVD prime2, v2
MOVD $0, v3
NEG prime1, v4
blockLoop()
ROR $64-1, v1, x1
ROR $64-7, v2, x2
ADD x1, x2
ROR $64-12, v3, x3
ROR $64-18, v4, x4
ADD x3, x4
ADD x2, x4, h
mergeRound(h, v1)
mergeRound(h, v2)
mergeRound(h, v3)
mergeRound(h, v4)
afterLoop:
ADD n, h
TBZ $4, n, try8
LDP.P 16(p), (x1, x2)
round0(x1)
// NOTE: here and below, sequencing the EOR after the ROR (using a
// rotated register) is worth a small but measurable speedup for small
// inputs.
ROR $64-27, h
EOR x1 @> 64-27, h, h
MADD h, prime4, prime1, h
round0(x2)
ROR $64-27, h
EOR x2 @> 64-27, h, h
MADD h, prime4, prime1, h
try8:
TBZ $3, n, try4
MOVD.P 8(p), x1
round0(x1)
ROR $64-27, h
EOR x1 @> 64-27, h, h
MADD h, prime4, prime1, h
try4:
TBZ $2, n, try2
MOVWU.P 4(p), x2
MUL prime1, x2
ROR $64-23, h
EOR x2 @> 64-23, h, h
MADD h, prime3, prime2, h
try2:
TBZ $1, n, try1
MOVHU.P 2(p), x3
AND $255, x3, x1
LSR $8, x3, x2
MUL prime5, x1
ROR $64-11, h
EOR x1 @> 64-11, h, h
MUL prime1, h
MUL prime5, x2
ROR $64-11, h
EOR x2 @> 64-11, h, h
MUL prime1, h
try1:
TBZ $0, n, finalize
MOVBU (p), x4
MUL prime5, x4
ROR $64-11, h
EOR x4 @> 64-11, h, h
MUL prime1, h
finalize:
EOR h >> 33, h
MUL prime2, h
EOR h >> 29, h
MUL prime3, h
EOR h >> 32, h
MOVD h, ret+24(FP)
RET
// func writeBlocks(d *Digest, b []byte) int
TEXT ·writeBlocks(SB), NOSPLIT|NOFRAME, $0-40
LDP ·primes+0(SB), (prime1, prime2)
// Load state. Assume v[1-4] are stored contiguously.
MOVD d+0(FP), digest
LDP 0(digest), (v1, v2)
LDP 16(digest), (v3, v4)
LDP b_base+8(FP), (p, n)
blockLoop()
// Store updated state.
STP (v1, v2), 0(digest)
STP (v3, v4), 16(digest)
BIC $31, n
MOVD n, ret+32(FP)
RET

View File

@@ -1,15 +0,0 @@
//go:build (amd64 || arm64) && !appengine && gc && !purego
// +build amd64 arm64
// +build !appengine
// +build gc
// +build !purego
package xxhash
// Sum64 computes the 64-bit xxHash digest of b.
//
//go:noescape
func Sum64(b []byte) uint64
//go:noescape
func writeBlocks(d *Digest, b []byte) int

View File

@@ -1,76 +0,0 @@
//go:build (!amd64 && !arm64) || appengine || !gc || purego
// +build !amd64,!arm64 appengine !gc purego
package xxhash
// Sum64 computes the 64-bit xxHash digest of b.
func Sum64(b []byte) uint64 {
// A simpler version would be
// d := New()
// d.Write(b)
// return d.Sum64()
// but this is faster, particularly for small inputs.
n := len(b)
var h uint64
if n >= 32 {
v1 := primes[0] + prime2
v2 := prime2
v3 := uint64(0)
v4 := -primes[0]
for len(b) >= 32 {
v1 = round(v1, u64(b[0:8:len(b)]))
v2 = round(v2, u64(b[8:16:len(b)]))
v3 = round(v3, u64(b[16:24:len(b)]))
v4 = round(v4, u64(b[24:32:len(b)]))
b = b[32:len(b):len(b)]
}
h = rol1(v1) + rol7(v2) + rol12(v3) + rol18(v4)
h = mergeRound(h, v1)
h = mergeRound(h, v2)
h = mergeRound(h, v3)
h = mergeRound(h, v4)
} else {
h = prime5
}
h += uint64(n)
for ; len(b) >= 8; b = b[8:] {
k1 := round(0, u64(b[:8]))
h ^= k1
h = rol27(h)*prime1 + prime4
}
if len(b) >= 4 {
h ^= uint64(u32(b[:4])) * prime1
h = rol23(h)*prime2 + prime3
b = b[4:]
}
for ; len(b) > 0; b = b[1:] {
h ^= uint64(b[0]) * prime5
h = rol11(h) * prime1
}
h ^= h >> 33
h *= prime2
h ^= h >> 29
h *= prime3
h ^= h >> 32
return h
}
func writeBlocks(d *Digest, b []byte) int {
v1, v2, v3, v4 := d.v1, d.v2, d.v3, d.v4
n := len(b)
for len(b) >= 32 {
v1 = round(v1, u64(b[0:8:len(b)]))
v2 = round(v2, u64(b[8:16:len(b)]))
v3 = round(v3, u64(b[16:24:len(b)]))
v4 = round(v4, u64(b[24:32:len(b)]))
b = b[32:len(b):len(b)]
}
d.v1, d.v2, d.v3, d.v4 = v1, v2, v3, v4
return n - len(b)
}

View File

@@ -1,16 +0,0 @@
//go:build appengine
// +build appengine
// This file contains the safe implementations of otherwise unsafe-using code.
package xxhash
// Sum64String computes the 64-bit xxHash digest of s.
func Sum64String(s string) uint64 {
return Sum64([]byte(s))
}
// WriteString adds more data to d. It always returns len(s), nil.
func (d *Digest) WriteString(s string) (n int, err error) {
return d.Write([]byte(s))
}

View File

@@ -1,58 +0,0 @@
//go:build !appengine
// +build !appengine
// This file encapsulates usage of unsafe.
// xxhash_safe.go contains the safe implementations.
package xxhash
import (
"unsafe"
)
// In the future it's possible that compiler optimizations will make these
// XxxString functions unnecessary by realizing that calls such as
// Sum64([]byte(s)) don't need to copy s. See https://go.dev/issue/2205.
// If that happens, even if we keep these functions they can be replaced with
// the trivial safe code.
// NOTE: The usual way of doing an unsafe string-to-[]byte conversion is:
//
// var b []byte
// bh := (*reflect.SliceHeader)(unsafe.Pointer(&b))
// bh.Data = (*reflect.StringHeader)(unsafe.Pointer(&s)).Data
// bh.Len = len(s)
// bh.Cap = len(s)
//
// Unfortunately, as of Go 1.15.3 the inliner's cost model assigns a high enough
// weight to this sequence of expressions that any function that uses it will
// not be inlined. Instead, the functions below use a different unsafe
// conversion designed to minimize the inliner weight and allow both to be
// inlined. There is also a test (TestInlining) which verifies that these are
// inlined.
//
// See https://github.com/golang/go/issues/42739 for discussion.
// Sum64String computes the 64-bit xxHash digest of s.
// It may be faster than Sum64([]byte(s)) by avoiding a copy.
func Sum64String(s string) uint64 {
b := *(*[]byte)(unsafe.Pointer(&sliceHeader{s, len(s)}))
return Sum64(b)
}
// WriteString adds more data to d. It always returns len(s), nil.
// It may be faster than Write([]byte(s)) by avoiding a copy.
func (d *Digest) WriteString(s string) (n int, err error) {
d.Write(*(*[]byte)(unsafe.Pointer(&sliceHeader{s, len(s)})))
// d.Write always returns len(s), nil.
// Ignoring the return output and returning these fixed values buys a
// savings of 6 in the inliner's cost model.
return len(s), nil
}
// sliceHeader is similar to reflect.SliceHeader, but it assumes that the layout
// of the first two words is the same as the layout of a string.
type sliceHeader struct {
s string
cap int
}

View File

@@ -1,15 +0,0 @@
ISC License
Copyright (c) 2012-2016 Dave Collins <dave@davec.name>
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

View File

@@ -1,145 +0,0 @@
// Copyright (c) 2015-2016 Dave Collins <dave@davec.name>
//
// Permission to use, copy, modify, and distribute this software for any
// purpose with or without fee is hereby granted, provided that the above
// copyright notice and this permission notice appear in all copies.
//
// THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
// WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
// MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
// ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
// WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
// ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
// OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
// NOTE: Due to the following build constraints, this file will only be compiled
// when the code is not running on Google App Engine, compiled by GopherJS, and
// "-tags safe" is not added to the go build command line. The "disableunsafe"
// tag is deprecated and thus should not be used.
// Go versions prior to 1.4 are disabled because they use a different layout
// for interfaces which make the implementation of unsafeReflectValue more complex.
// +build !js,!appengine,!safe,!disableunsafe,go1.4
package spew
import (
"reflect"
"unsafe"
)
const (
// UnsafeDisabled is a build-time constant which specifies whether or
// not access to the unsafe package is available.
UnsafeDisabled = false
// ptrSize is the size of a pointer on the current arch.
ptrSize = unsafe.Sizeof((*byte)(nil))
)
type flag uintptr
var (
// flagRO indicates whether the value field of a reflect.Value
// is read-only.
flagRO flag
// flagAddr indicates whether the address of the reflect.Value's
// value may be taken.
flagAddr flag
)
// flagKindMask holds the bits that make up the kind
// part of the flags field. In all the supported versions,
// it is in the lower 5 bits.
const flagKindMask = flag(0x1f)
// Different versions of Go have used different
// bit layouts for the flags type. This table
// records the known combinations.
var okFlags = []struct {
ro, addr flag
}{{
// From Go 1.4 to 1.5
ro: 1 << 5,
addr: 1 << 7,
}, {
// Up to Go tip.
ro: 1<<5 | 1<<6,
addr: 1 << 8,
}}
var flagValOffset = func() uintptr {
field, ok := reflect.TypeOf(reflect.Value{}).FieldByName("flag")
if !ok {
panic("reflect.Value has no flag field")
}
return field.Offset
}()
// flagField returns a pointer to the flag field of a reflect.Value.
func flagField(v *reflect.Value) *flag {
return (*flag)(unsafe.Pointer(uintptr(unsafe.Pointer(v)) + flagValOffset))
}
// unsafeReflectValue converts the passed reflect.Value into a one that bypasses
// the typical safety restrictions preventing access to unaddressable and
// unexported data. It works by digging the raw pointer to the underlying
// value out of the protected value and generating a new unprotected (unsafe)
// reflect.Value to it.
//
// This allows us to check for implementations of the Stringer and error
// interfaces to be used for pretty printing ordinarily unaddressable and
// inaccessible values such as unexported struct fields.
func unsafeReflectValue(v reflect.Value) reflect.Value {
if !v.IsValid() || (v.CanInterface() && v.CanAddr()) {
return v
}
flagFieldPtr := flagField(&v)
*flagFieldPtr &^= flagRO
*flagFieldPtr |= flagAddr
return v
}
// Sanity checks against future reflect package changes
// to the type or semantics of the Value.flag field.
func init() {
field, ok := reflect.TypeOf(reflect.Value{}).FieldByName("flag")
if !ok {
panic("reflect.Value has no flag field")
}
if field.Type.Kind() != reflect.TypeOf(flag(0)).Kind() {
panic("reflect.Value flag field has changed kind")
}
type t0 int
var t struct {
A t0
// t0 will have flagEmbedRO set.
t0
// a will have flagStickyRO set
a t0
}
vA := reflect.ValueOf(t).FieldByName("A")
va := reflect.ValueOf(t).FieldByName("a")
vt0 := reflect.ValueOf(t).FieldByName("t0")
// Infer flagRO from the difference between the flags
// for the (otherwise identical) fields in t.
flagPublic := *flagField(&vA)
flagWithRO := *flagField(&va) | *flagField(&vt0)
flagRO = flagPublic ^ flagWithRO
// Infer flagAddr from the difference between a value
// taken from a pointer and not.
vPtrA := reflect.ValueOf(&t).Elem().FieldByName("A")
flagNoPtr := *flagField(&vA)
flagPtr := *flagField(&vPtrA)
flagAddr = flagNoPtr ^ flagPtr
// Check that the inferred flags tally with one of the known versions.
for _, f := range okFlags {
if flagRO == f.ro && flagAddr == f.addr {
return
}
}
panic("reflect.Value read-only flag has changed semantics")
}

View File

@@ -1,38 +0,0 @@
// Copyright (c) 2015-2016 Dave Collins <dave@davec.name>
//
// Permission to use, copy, modify, and distribute this software for any
// purpose with or without fee is hereby granted, provided that the above
// copyright notice and this permission notice appear in all copies.
//
// THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
// WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
// MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
// ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
// WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
// ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
// OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
// NOTE: Due to the following build constraints, this file will only be compiled
// when the code is running on Google App Engine, compiled by GopherJS, or
// "-tags safe" is added to the go build command line. The "disableunsafe"
// tag is deprecated and thus should not be used.
// +build js appengine safe disableunsafe !go1.4
package spew
import "reflect"
const (
// UnsafeDisabled is a build-time constant which specifies whether or
// not access to the unsafe package is available.
UnsafeDisabled = true
)
// unsafeReflectValue typically converts the passed reflect.Value into a one
// that bypasses the typical safety restrictions preventing access to
// unaddressable and unexported data. However, doing this relies on access to
// the unsafe package. This is a stub version which simply returns the passed
// reflect.Value when the unsafe package is not available.
func unsafeReflectValue(v reflect.Value) reflect.Value {
return v
}

View File

@@ -1,341 +0,0 @@
/*
* Copyright (c) 2013-2016 Dave Collins <dave@davec.name>
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
package spew
import (
"bytes"
"fmt"
"io"
"reflect"
"sort"
"strconv"
)
// Some constants in the form of bytes to avoid string overhead. This mirrors
// the technique used in the fmt package.
var (
panicBytes = []byte("(PANIC=")
plusBytes = []byte("+")
iBytes = []byte("i")
trueBytes = []byte("true")
falseBytes = []byte("false")
interfaceBytes = []byte("(interface {})")
commaNewlineBytes = []byte(",\n")
newlineBytes = []byte("\n")
openBraceBytes = []byte("{")
openBraceNewlineBytes = []byte("{\n")
closeBraceBytes = []byte("}")
asteriskBytes = []byte("*")
colonBytes = []byte(":")
colonSpaceBytes = []byte(": ")
openParenBytes = []byte("(")
closeParenBytes = []byte(")")
spaceBytes = []byte(" ")
pointerChainBytes = []byte("->")
nilAngleBytes = []byte("<nil>")
maxNewlineBytes = []byte("<max depth reached>\n")
maxShortBytes = []byte("<max>")
circularBytes = []byte("<already shown>")
circularShortBytes = []byte("<shown>")
invalidAngleBytes = []byte("<invalid>")
openBracketBytes = []byte("[")
closeBracketBytes = []byte("]")
percentBytes = []byte("%")
precisionBytes = []byte(".")
openAngleBytes = []byte("<")
closeAngleBytes = []byte(">")
openMapBytes = []byte("map[")
closeMapBytes = []byte("]")
lenEqualsBytes = []byte("len=")
capEqualsBytes = []byte("cap=")
)
// hexDigits is used to map a decimal value to a hex digit.
var hexDigits = "0123456789abcdef"
// catchPanic handles any panics that might occur during the handleMethods
// calls.
func catchPanic(w io.Writer, v reflect.Value) {
if err := recover(); err != nil {
w.Write(panicBytes)
fmt.Fprintf(w, "%v", err)
w.Write(closeParenBytes)
}
}
// handleMethods attempts to call the Error and String methods on the underlying
// type the passed reflect.Value represents and outputes the result to Writer w.
//
// It handles panics in any called methods by catching and displaying the error
// as the formatted value.
func handleMethods(cs *ConfigState, w io.Writer, v reflect.Value) (handled bool) {
// We need an interface to check if the type implements the error or
// Stringer interface. However, the reflect package won't give us an
// interface on certain things like unexported struct fields in order
// to enforce visibility rules. We use unsafe, when it's available,
// to bypass these restrictions since this package does not mutate the
// values.
if !v.CanInterface() {
if UnsafeDisabled {
return false
}
v = unsafeReflectValue(v)
}
// Choose whether or not to do error and Stringer interface lookups against
// the base type or a pointer to the base type depending on settings.
// Technically calling one of these methods with a pointer receiver can
// mutate the value, however, types which choose to satisify an error or
// Stringer interface with a pointer receiver should not be mutating their
// state inside these interface methods.
if !cs.DisablePointerMethods && !UnsafeDisabled && !v.CanAddr() {
v = unsafeReflectValue(v)
}
if v.CanAddr() {
v = v.Addr()
}
// Is it an error or Stringer?
switch iface := v.Interface().(type) {
case error:
defer catchPanic(w, v)
if cs.ContinueOnMethod {
w.Write(openParenBytes)
w.Write([]byte(iface.Error()))
w.Write(closeParenBytes)
w.Write(spaceBytes)
return false
}
w.Write([]byte(iface.Error()))
return true
case fmt.Stringer:
defer catchPanic(w, v)
if cs.ContinueOnMethod {
w.Write(openParenBytes)
w.Write([]byte(iface.String()))
w.Write(closeParenBytes)
w.Write(spaceBytes)
return false
}
w.Write([]byte(iface.String()))
return true
}
return false
}
// printBool outputs a boolean value as true or false to Writer w.
func printBool(w io.Writer, val bool) {
if val {
w.Write(trueBytes)
} else {
w.Write(falseBytes)
}
}
// printInt outputs a signed integer value to Writer w.
func printInt(w io.Writer, val int64, base int) {
w.Write([]byte(strconv.FormatInt(val, base)))
}
// printUint outputs an unsigned integer value to Writer w.
func printUint(w io.Writer, val uint64, base int) {
w.Write([]byte(strconv.FormatUint(val, base)))
}
// printFloat outputs a floating point value using the specified precision,
// which is expected to be 32 or 64bit, to Writer w.
func printFloat(w io.Writer, val float64, precision int) {
w.Write([]byte(strconv.FormatFloat(val, 'g', -1, precision)))
}
// printComplex outputs a complex value using the specified float precision
// for the real and imaginary parts to Writer w.
func printComplex(w io.Writer, c complex128, floatPrecision int) {
r := real(c)
w.Write(openParenBytes)
w.Write([]byte(strconv.FormatFloat(r, 'g', -1, floatPrecision)))
i := imag(c)
if i >= 0 {
w.Write(plusBytes)
}
w.Write([]byte(strconv.FormatFloat(i, 'g', -1, floatPrecision)))
w.Write(iBytes)
w.Write(closeParenBytes)
}
// printHexPtr outputs a uintptr formatted as hexadecimal with a leading '0x'
// prefix to Writer w.
func printHexPtr(w io.Writer, p uintptr) {
// Null pointer.
num := uint64(p)
if num == 0 {
w.Write(nilAngleBytes)
return
}
// Max uint64 is 16 bytes in hex + 2 bytes for '0x' prefix
buf := make([]byte, 18)
// It's simpler to construct the hex string right to left.
base := uint64(16)
i := len(buf) - 1
for num >= base {
buf[i] = hexDigits[num%base]
num /= base
i--
}
buf[i] = hexDigits[num]
// Add '0x' prefix.
i--
buf[i] = 'x'
i--
buf[i] = '0'
// Strip unused leading bytes.
buf = buf[i:]
w.Write(buf)
}
// valuesSorter implements sort.Interface to allow a slice of reflect.Value
// elements to be sorted.
type valuesSorter struct {
values []reflect.Value
strings []string // either nil or same len and values
cs *ConfigState
}
// newValuesSorter initializes a valuesSorter instance, which holds a set of
// surrogate keys on which the data should be sorted. It uses flags in
// ConfigState to decide if and how to populate those surrogate keys.
func newValuesSorter(values []reflect.Value, cs *ConfigState) sort.Interface {
vs := &valuesSorter{values: values, cs: cs}
if canSortSimply(vs.values[0].Kind()) {
return vs
}
if !cs.DisableMethods {
vs.strings = make([]string, len(values))
for i := range vs.values {
b := bytes.Buffer{}
if !handleMethods(cs, &b, vs.values[i]) {
vs.strings = nil
break
}
vs.strings[i] = b.String()
}
}
if vs.strings == nil && cs.SpewKeys {
vs.strings = make([]string, len(values))
for i := range vs.values {
vs.strings[i] = Sprintf("%#v", vs.values[i].Interface())
}
}
return vs
}
// canSortSimply tests whether a reflect.Kind is a primitive that can be sorted
// directly, or whether it should be considered for sorting by surrogate keys
// (if the ConfigState allows it).
func canSortSimply(kind reflect.Kind) bool {
// This switch parallels valueSortLess, except for the default case.
switch kind {
case reflect.Bool:
return true
case reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Int:
return true
case reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uint:
return true
case reflect.Float32, reflect.Float64:
return true
case reflect.String:
return true
case reflect.Uintptr:
return true
case reflect.Array:
return true
}
return false
}
// Len returns the number of values in the slice. It is part of the
// sort.Interface implementation.
func (s *valuesSorter) Len() int {
return len(s.values)
}
// Swap swaps the values at the passed indices. It is part of the
// sort.Interface implementation.
func (s *valuesSorter) Swap(i, j int) {
s.values[i], s.values[j] = s.values[j], s.values[i]
if s.strings != nil {
s.strings[i], s.strings[j] = s.strings[j], s.strings[i]
}
}
// valueSortLess returns whether the first value should sort before the second
// value. It is used by valueSorter.Less as part of the sort.Interface
// implementation.
func valueSortLess(a, b reflect.Value) bool {
switch a.Kind() {
case reflect.Bool:
return !a.Bool() && b.Bool()
case reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Int:
return a.Int() < b.Int()
case reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uint:
return a.Uint() < b.Uint()
case reflect.Float32, reflect.Float64:
return a.Float() < b.Float()
case reflect.String:
return a.String() < b.String()
case reflect.Uintptr:
return a.Uint() < b.Uint()
case reflect.Array:
// Compare the contents of both arrays.
l := a.Len()
for i := 0; i < l; i++ {
av := a.Index(i)
bv := b.Index(i)
if av.Interface() == bv.Interface() {
continue
}
return valueSortLess(av, bv)
}
}
return a.String() < b.String()
}
// Less returns whether the value at index i should sort before the
// value at index j. It is part of the sort.Interface implementation.
func (s *valuesSorter) Less(i, j int) bool {
if s.strings == nil {
return valueSortLess(s.values[i], s.values[j])
}
return s.strings[i] < s.strings[j]
}
// sortValues is a sort function that handles both native types and any type that
// can be converted to error or Stringer. Other inputs are sorted according to
// their Value.String() value to ensure display stability.
func sortValues(values []reflect.Value, cs *ConfigState) {
if len(values) == 0 {
return
}
sort.Sort(newValuesSorter(values, cs))
}

View File

@@ -1,306 +0,0 @@
/*
* Copyright (c) 2013-2016 Dave Collins <dave@davec.name>
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
package spew
import (
"bytes"
"fmt"
"io"
"os"
)
// ConfigState houses the configuration options used by spew to format and
// display values. There is a global instance, Config, that is used to control
// all top-level Formatter and Dump functionality. Each ConfigState instance
// provides methods equivalent to the top-level functions.
//
// The zero value for ConfigState provides no indentation. You would typically
// want to set it to a space or a tab.
//
// Alternatively, you can use NewDefaultConfig to get a ConfigState instance
// with default settings. See the documentation of NewDefaultConfig for default
// values.
type ConfigState struct {
// Indent specifies the string to use for each indentation level. The
// global config instance that all top-level functions use set this to a
// single space by default. If you would like more indentation, you might
// set this to a tab with "\t" or perhaps two spaces with " ".
Indent string
// MaxDepth controls the maximum number of levels to descend into nested
// data structures. The default, 0, means there is no limit.
//
// NOTE: Circular data structures are properly detected, so it is not
// necessary to set this value unless you specifically want to limit deeply
// nested data structures.
MaxDepth int
// DisableMethods specifies whether or not error and Stringer interfaces are
// invoked for types that implement them.
DisableMethods bool
// DisablePointerMethods specifies whether or not to check for and invoke
// error and Stringer interfaces on types which only accept a pointer
// receiver when the current type is not a pointer.
//
// NOTE: This might be an unsafe action since calling one of these methods
// with a pointer receiver could technically mutate the value, however,
// in practice, types which choose to satisify an error or Stringer
// interface with a pointer receiver should not be mutating their state
// inside these interface methods. As a result, this option relies on
// access to the unsafe package, so it will not have any effect when
// running in environments without access to the unsafe package such as
// Google App Engine or with the "safe" build tag specified.
DisablePointerMethods bool
// DisablePointerAddresses specifies whether to disable the printing of
// pointer addresses. This is useful when diffing data structures in tests.
DisablePointerAddresses bool
// DisableCapacities specifies whether to disable the printing of capacities
// for arrays, slices, maps and channels. This is useful when diffing
// data structures in tests.
DisableCapacities bool
// ContinueOnMethod specifies whether or not recursion should continue once
// a custom error or Stringer interface is invoked. The default, false,
// means it will print the results of invoking the custom error or Stringer
// interface and return immediately instead of continuing to recurse into
// the internals of the data type.
//
// NOTE: This flag does not have any effect if method invocation is disabled
// via the DisableMethods or DisablePointerMethods options.
ContinueOnMethod bool
// SortKeys specifies map keys should be sorted before being printed. Use
// this to have a more deterministic, diffable output. Note that only
// native types (bool, int, uint, floats, uintptr and string) and types
// that support the error or Stringer interfaces (if methods are
// enabled) are supported, with other types sorted according to the
// reflect.Value.String() output which guarantees display stability.
SortKeys bool
// SpewKeys specifies that, as a last resort attempt, map keys should
// be spewed to strings and sorted by those strings. This is only
// considered if SortKeys is true.
SpewKeys bool
}
// Config is the active configuration of the top-level functions.
// The configuration can be changed by modifying the contents of spew.Config.
var Config = ConfigState{Indent: " "}
// Errorf is a wrapper for fmt.Errorf that treats each argument as if it were
// passed with a Formatter interface returned by c.NewFormatter. It returns
// the formatted string as a value that satisfies error. See NewFormatter
// for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Errorf(format, c.NewFormatter(a), c.NewFormatter(b))
func (c *ConfigState) Errorf(format string, a ...interface{}) (err error) {
return fmt.Errorf(format, c.convertArgs(a)...)
}
// Fprint is a wrapper for fmt.Fprint that treats each argument as if it were
// passed with a Formatter interface returned by c.NewFormatter. It returns
// the number of bytes written and any write error encountered. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Fprint(w, c.NewFormatter(a), c.NewFormatter(b))
func (c *ConfigState) Fprint(w io.Writer, a ...interface{}) (n int, err error) {
return fmt.Fprint(w, c.convertArgs(a)...)
}
// Fprintf is a wrapper for fmt.Fprintf that treats each argument as if it were
// passed with a Formatter interface returned by c.NewFormatter. It returns
// the number of bytes written and any write error encountered. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Fprintf(w, format, c.NewFormatter(a), c.NewFormatter(b))
func (c *ConfigState) Fprintf(w io.Writer, format string, a ...interface{}) (n int, err error) {
return fmt.Fprintf(w, format, c.convertArgs(a)...)
}
// Fprintln is a wrapper for fmt.Fprintln that treats each argument as if it
// passed with a Formatter interface returned by c.NewFormatter. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Fprintln(w, c.NewFormatter(a), c.NewFormatter(b))
func (c *ConfigState) Fprintln(w io.Writer, a ...interface{}) (n int, err error) {
return fmt.Fprintln(w, c.convertArgs(a)...)
}
// Print is a wrapper for fmt.Print that treats each argument as if it were
// passed with a Formatter interface returned by c.NewFormatter. It returns
// the number of bytes written and any write error encountered. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Print(c.NewFormatter(a), c.NewFormatter(b))
func (c *ConfigState) Print(a ...interface{}) (n int, err error) {
return fmt.Print(c.convertArgs(a)...)
}
// Printf is a wrapper for fmt.Printf that treats each argument as if it were
// passed with a Formatter interface returned by c.NewFormatter. It returns
// the number of bytes written and any write error encountered. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Printf(format, c.NewFormatter(a), c.NewFormatter(b))
func (c *ConfigState) Printf(format string, a ...interface{}) (n int, err error) {
return fmt.Printf(format, c.convertArgs(a)...)
}
// Println is a wrapper for fmt.Println that treats each argument as if it were
// passed with a Formatter interface returned by c.NewFormatter. It returns
// the number of bytes written and any write error encountered. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Println(c.NewFormatter(a), c.NewFormatter(b))
func (c *ConfigState) Println(a ...interface{}) (n int, err error) {
return fmt.Println(c.convertArgs(a)...)
}
// Sprint is a wrapper for fmt.Sprint that treats each argument as if it were
// passed with a Formatter interface returned by c.NewFormatter. It returns
// the resulting string. See NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Sprint(c.NewFormatter(a), c.NewFormatter(b))
func (c *ConfigState) Sprint(a ...interface{}) string {
return fmt.Sprint(c.convertArgs(a)...)
}
// Sprintf is a wrapper for fmt.Sprintf that treats each argument as if it were
// passed with a Formatter interface returned by c.NewFormatter. It returns
// the resulting string. See NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Sprintf(format, c.NewFormatter(a), c.NewFormatter(b))
func (c *ConfigState) Sprintf(format string, a ...interface{}) string {
return fmt.Sprintf(format, c.convertArgs(a)...)
}
// Sprintln is a wrapper for fmt.Sprintln that treats each argument as if it
// were passed with a Formatter interface returned by c.NewFormatter. It
// returns the resulting string. See NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Sprintln(c.NewFormatter(a), c.NewFormatter(b))
func (c *ConfigState) Sprintln(a ...interface{}) string {
return fmt.Sprintln(c.convertArgs(a)...)
}
/*
NewFormatter returns a custom formatter that satisfies the fmt.Formatter
interface. As a result, it integrates cleanly with standard fmt package
printing functions. The formatter is useful for inline printing of smaller data
types similar to the standard %v format specifier.
The custom formatter only responds to the %v (most compact), %+v (adds pointer
addresses), %#v (adds types), and %#+v (adds types and pointer addresses) verb
combinations. Any other verbs such as %x and %q will be sent to the the
standard fmt package for formatting. In addition, the custom formatter ignores
the width and precision arguments (however they will still work on the format
specifiers not handled by the custom formatter).
Typically this function shouldn't be called directly. It is much easier to make
use of the custom formatter by calling one of the convenience functions such as
c.Printf, c.Println, or c.Printf.
*/
func (c *ConfigState) NewFormatter(v interface{}) fmt.Formatter {
return newFormatter(c, v)
}
// Fdump formats and displays the passed arguments to io.Writer w. It formats
// exactly the same as Dump.
func (c *ConfigState) Fdump(w io.Writer, a ...interface{}) {
fdump(c, w, a...)
}
/*
Dump displays the passed parameters to standard out with newlines, customizable
indentation, and additional debug information such as complete types and all
pointer addresses used to indirect to the final value. It provides the
following features over the built-in printing facilities provided by the fmt
package:
* Pointers are dereferenced and followed
* Circular data structures are detected and handled properly
* Custom Stringer/error interfaces are optionally invoked, including
on unexported types
* Custom types which only implement the Stringer/error interfaces via
a pointer receiver are optionally invoked when passing non-pointer
variables
* Byte arrays and slices are dumped like the hexdump -C command which
includes offsets, byte values in hex, and ASCII output
The configuration options are controlled by modifying the public members
of c. See ConfigState for options documentation.
See Fdump if you would prefer dumping to an arbitrary io.Writer or Sdump to
get the formatted result as a string.
*/
func (c *ConfigState) Dump(a ...interface{}) {
fdump(c, os.Stdout, a...)
}
// Sdump returns a string with the passed arguments formatted exactly the same
// as Dump.
func (c *ConfigState) Sdump(a ...interface{}) string {
var buf bytes.Buffer
fdump(c, &buf, a...)
return buf.String()
}
// convertArgs accepts a slice of arguments and returns a slice of the same
// length with each argument converted to a spew Formatter interface using
// the ConfigState associated with s.
func (c *ConfigState) convertArgs(args []interface{}) (formatters []interface{}) {
formatters = make([]interface{}, len(args))
for index, arg := range args {
formatters[index] = newFormatter(c, arg)
}
return formatters
}
// NewDefaultConfig returns a ConfigState with the following default settings.
//
// Indent: " "
// MaxDepth: 0
// DisableMethods: false
// DisablePointerMethods: false
// ContinueOnMethod: false
// SortKeys: false
func NewDefaultConfig() *ConfigState {
return &ConfigState{Indent: " "}
}

View File

@@ -1,211 +0,0 @@
/*
* Copyright (c) 2013-2016 Dave Collins <dave@davec.name>
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
/*
Package spew implements a deep pretty printer for Go data structures to aid in
debugging.
A quick overview of the additional features spew provides over the built-in
printing facilities for Go data types are as follows:
* Pointers are dereferenced and followed
* Circular data structures are detected and handled properly
* Custom Stringer/error interfaces are optionally invoked, including
on unexported types
* Custom types which only implement the Stringer/error interfaces via
a pointer receiver are optionally invoked when passing non-pointer
variables
* Byte arrays and slices are dumped like the hexdump -C command which
includes offsets, byte values in hex, and ASCII output (only when using
Dump style)
There are two different approaches spew allows for dumping Go data structures:
* Dump style which prints with newlines, customizable indentation,
and additional debug information such as types and all pointer addresses
used to indirect to the final value
* A custom Formatter interface that integrates cleanly with the standard fmt
package and replaces %v, %+v, %#v, and %#+v to provide inline printing
similar to the default %v while providing the additional functionality
outlined above and passing unsupported format verbs such as %x and %q
along to fmt
Quick Start
This section demonstrates how to quickly get started with spew. See the
sections below for further details on formatting and configuration options.
To dump a variable with full newlines, indentation, type, and pointer
information use Dump, Fdump, or Sdump:
spew.Dump(myVar1, myVar2, ...)
spew.Fdump(someWriter, myVar1, myVar2, ...)
str := spew.Sdump(myVar1, myVar2, ...)
Alternatively, if you would prefer to use format strings with a compacted inline
printing style, use the convenience wrappers Printf, Fprintf, etc with
%v (most compact), %+v (adds pointer addresses), %#v (adds types), or
%#+v (adds types and pointer addresses):
spew.Printf("myVar1: %v -- myVar2: %+v", myVar1, myVar2)
spew.Printf("myVar3: %#v -- myVar4: %#+v", myVar3, myVar4)
spew.Fprintf(someWriter, "myVar1: %v -- myVar2: %+v", myVar1, myVar2)
spew.Fprintf(someWriter, "myVar3: %#v -- myVar4: %#+v", myVar3, myVar4)
Configuration Options
Configuration of spew is handled by fields in the ConfigState type. For
convenience, all of the top-level functions use a global state available
via the spew.Config global.
It is also possible to create a ConfigState instance that provides methods
equivalent to the top-level functions. This allows concurrent configuration
options. See the ConfigState documentation for more details.
The following configuration options are available:
* Indent
String to use for each indentation level for Dump functions.
It is a single space by default. A popular alternative is "\t".
* MaxDepth
Maximum number of levels to descend into nested data structures.
There is no limit by default.
* DisableMethods
Disables invocation of error and Stringer interface methods.
Method invocation is enabled by default.
* DisablePointerMethods
Disables invocation of error and Stringer interface methods on types
which only accept pointer receivers from non-pointer variables.
Pointer method invocation is enabled by default.
* DisablePointerAddresses
DisablePointerAddresses specifies whether to disable the printing of
pointer addresses. This is useful when diffing data structures in tests.
* DisableCapacities
DisableCapacities specifies whether to disable the printing of
capacities for arrays, slices, maps and channels. This is useful when
diffing data structures in tests.
* ContinueOnMethod
Enables recursion into types after invoking error and Stringer interface
methods. Recursion after method invocation is disabled by default.
* SortKeys
Specifies map keys should be sorted before being printed. Use
this to have a more deterministic, diffable output. Note that
only native types (bool, int, uint, floats, uintptr and string)
and types which implement error or Stringer interfaces are
supported with other types sorted according to the
reflect.Value.String() output which guarantees display
stability. Natural map order is used by default.
* SpewKeys
Specifies that, as a last resort attempt, map keys should be
spewed to strings and sorted by those strings. This is only
considered if SortKeys is true.
Dump Usage
Simply call spew.Dump with a list of variables you want to dump:
spew.Dump(myVar1, myVar2, ...)
You may also call spew.Fdump if you would prefer to output to an arbitrary
io.Writer. For example, to dump to standard error:
spew.Fdump(os.Stderr, myVar1, myVar2, ...)
A third option is to call spew.Sdump to get the formatted output as a string:
str := spew.Sdump(myVar1, myVar2, ...)
Sample Dump Output
See the Dump example for details on the setup of the types and variables being
shown here.
(main.Foo) {
unexportedField: (*main.Bar)(0xf84002e210)({
flag: (main.Flag) flagTwo,
data: (uintptr) <nil>
}),
ExportedField: (map[interface {}]interface {}) (len=1) {
(string) (len=3) "one": (bool) true
}
}
Byte (and uint8) arrays and slices are displayed uniquely like the hexdump -C
command as shown.
([]uint8) (len=32 cap=32) {
00000000 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 |............... |
00000010 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f 30 |!"#$%&'()*+,-./0|
00000020 31 32 |12|
}
Custom Formatter
Spew provides a custom formatter that implements the fmt.Formatter interface
so that it integrates cleanly with standard fmt package printing functions. The
formatter is useful for inline printing of smaller data types similar to the
standard %v format specifier.
The custom formatter only responds to the %v (most compact), %+v (adds pointer
addresses), %#v (adds types), or %#+v (adds types and pointer addresses) verb
combinations. Any other verbs such as %x and %q will be sent to the the
standard fmt package for formatting. In addition, the custom formatter ignores
the width and precision arguments (however they will still work on the format
specifiers not handled by the custom formatter).
Custom Formatter Usage
The simplest way to make use of the spew custom formatter is to call one of the
convenience functions such as spew.Printf, spew.Println, or spew.Printf. The
functions have syntax you are most likely already familiar with:
spew.Printf("myVar1: %v -- myVar2: %+v", myVar1, myVar2)
spew.Printf("myVar3: %#v -- myVar4: %#+v", myVar3, myVar4)
spew.Println(myVar, myVar2)
spew.Fprintf(os.Stderr, "myVar1: %v -- myVar2: %+v", myVar1, myVar2)
spew.Fprintf(os.Stderr, "myVar3: %#v -- myVar4: %#+v", myVar3, myVar4)
See the Index for the full list convenience functions.
Sample Formatter Output
Double pointer to a uint8:
%v: <**>5
%+v: <**>(0xf8400420d0->0xf8400420c8)5
%#v: (**uint8)5
%#+v: (**uint8)(0xf8400420d0->0xf8400420c8)5
Pointer to circular struct with a uint8 field and a pointer to itself:
%v: <*>{1 <*><shown>}
%+v: <*>(0xf84003e260){ui8:1 c:<*>(0xf84003e260)<shown>}
%#v: (*main.circular){ui8:(uint8)1 c:(*main.circular)<shown>}
%#+v: (*main.circular)(0xf84003e260){ui8:(uint8)1 c:(*main.circular)(0xf84003e260)<shown>}
See the Printf example for details on the setup of variables being shown
here.
Errors
Since it is possible for custom Stringer/error interfaces to panic, spew
detects them and handles them internally by printing the panic information
inline with the output. Since spew is intended to provide deep pretty printing
capabilities on structures, it intentionally does not return any errors.
*/
package spew

View File

@@ -1,509 +0,0 @@
/*
* Copyright (c) 2013-2016 Dave Collins <dave@davec.name>
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
package spew
import (
"bytes"
"encoding/hex"
"fmt"
"io"
"os"
"reflect"
"regexp"
"strconv"
"strings"
)
var (
// uint8Type is a reflect.Type representing a uint8. It is used to
// convert cgo types to uint8 slices for hexdumping.
uint8Type = reflect.TypeOf(uint8(0))
// cCharRE is a regular expression that matches a cgo char.
// It is used to detect character arrays to hexdump them.
cCharRE = regexp.MustCompile(`^.*\._Ctype_char$`)
// cUnsignedCharRE is a regular expression that matches a cgo unsigned
// char. It is used to detect unsigned character arrays to hexdump
// them.
cUnsignedCharRE = regexp.MustCompile(`^.*\._Ctype_unsignedchar$`)
// cUint8tCharRE is a regular expression that matches a cgo uint8_t.
// It is used to detect uint8_t arrays to hexdump them.
cUint8tCharRE = regexp.MustCompile(`^.*\._Ctype_uint8_t$`)
)
// dumpState contains information about the state of a dump operation.
type dumpState struct {
w io.Writer
depth int
pointers map[uintptr]int
ignoreNextType bool
ignoreNextIndent bool
cs *ConfigState
}
// indent performs indentation according to the depth level and cs.Indent
// option.
func (d *dumpState) indent() {
if d.ignoreNextIndent {
d.ignoreNextIndent = false
return
}
d.w.Write(bytes.Repeat([]byte(d.cs.Indent), d.depth))
}
// unpackValue returns values inside of non-nil interfaces when possible.
// This is useful for data types like structs, arrays, slices, and maps which
// can contain varying types packed inside an interface.
func (d *dumpState) unpackValue(v reflect.Value) reflect.Value {
if v.Kind() == reflect.Interface && !v.IsNil() {
v = v.Elem()
}
return v
}
// dumpPtr handles formatting of pointers by indirecting them as necessary.
func (d *dumpState) dumpPtr(v reflect.Value) {
// Remove pointers at or below the current depth from map used to detect
// circular refs.
for k, depth := range d.pointers {
if depth >= d.depth {
delete(d.pointers, k)
}
}
// Keep list of all dereferenced pointers to show later.
pointerChain := make([]uintptr, 0)
// Figure out how many levels of indirection there are by dereferencing
// pointers and unpacking interfaces down the chain while detecting circular
// references.
nilFound := false
cycleFound := false
indirects := 0
ve := v
for ve.Kind() == reflect.Ptr {
if ve.IsNil() {
nilFound = true
break
}
indirects++
addr := ve.Pointer()
pointerChain = append(pointerChain, addr)
if pd, ok := d.pointers[addr]; ok && pd < d.depth {
cycleFound = true
indirects--
break
}
d.pointers[addr] = d.depth
ve = ve.Elem()
if ve.Kind() == reflect.Interface {
if ve.IsNil() {
nilFound = true
break
}
ve = ve.Elem()
}
}
// Display type information.
d.w.Write(openParenBytes)
d.w.Write(bytes.Repeat(asteriskBytes, indirects))
d.w.Write([]byte(ve.Type().String()))
d.w.Write(closeParenBytes)
// Display pointer information.
if !d.cs.DisablePointerAddresses && len(pointerChain) > 0 {
d.w.Write(openParenBytes)
for i, addr := range pointerChain {
if i > 0 {
d.w.Write(pointerChainBytes)
}
printHexPtr(d.w, addr)
}
d.w.Write(closeParenBytes)
}
// Display dereferenced value.
d.w.Write(openParenBytes)
switch {
case nilFound:
d.w.Write(nilAngleBytes)
case cycleFound:
d.w.Write(circularBytes)
default:
d.ignoreNextType = true
d.dump(ve)
}
d.w.Write(closeParenBytes)
}
// dumpSlice handles formatting of arrays and slices. Byte (uint8 under
// reflection) arrays and slices are dumped in hexdump -C fashion.
func (d *dumpState) dumpSlice(v reflect.Value) {
// Determine whether this type should be hex dumped or not. Also,
// for types which should be hexdumped, try to use the underlying data
// first, then fall back to trying to convert them to a uint8 slice.
var buf []uint8
doConvert := false
doHexDump := false
numEntries := v.Len()
if numEntries > 0 {
vt := v.Index(0).Type()
vts := vt.String()
switch {
// C types that need to be converted.
case cCharRE.MatchString(vts):
fallthrough
case cUnsignedCharRE.MatchString(vts):
fallthrough
case cUint8tCharRE.MatchString(vts):
doConvert = true
// Try to use existing uint8 slices and fall back to converting
// and copying if that fails.
case vt.Kind() == reflect.Uint8:
// We need an addressable interface to convert the type
// to a byte slice. However, the reflect package won't
// give us an interface on certain things like
// unexported struct fields in order to enforce
// visibility rules. We use unsafe, when available, to
// bypass these restrictions since this package does not
// mutate the values.
vs := v
if !vs.CanInterface() || !vs.CanAddr() {
vs = unsafeReflectValue(vs)
}
if !UnsafeDisabled {
vs = vs.Slice(0, numEntries)
// Use the existing uint8 slice if it can be
// type asserted.
iface := vs.Interface()
if slice, ok := iface.([]uint8); ok {
buf = slice
doHexDump = true
break
}
}
// The underlying data needs to be converted if it can't
// be type asserted to a uint8 slice.
doConvert = true
}
// Copy and convert the underlying type if needed.
if doConvert && vt.ConvertibleTo(uint8Type) {
// Convert and copy each element into a uint8 byte
// slice.
buf = make([]uint8, numEntries)
for i := 0; i < numEntries; i++ {
vv := v.Index(i)
buf[i] = uint8(vv.Convert(uint8Type).Uint())
}
doHexDump = true
}
}
// Hexdump the entire slice as needed.
if doHexDump {
indent := strings.Repeat(d.cs.Indent, d.depth)
str := indent + hex.Dump(buf)
str = strings.Replace(str, "\n", "\n"+indent, -1)
str = strings.TrimRight(str, d.cs.Indent)
d.w.Write([]byte(str))
return
}
// Recursively call dump for each item.
for i := 0; i < numEntries; i++ {
d.dump(d.unpackValue(v.Index(i)))
if i < (numEntries - 1) {
d.w.Write(commaNewlineBytes)
} else {
d.w.Write(newlineBytes)
}
}
}
// dump is the main workhorse for dumping a value. It uses the passed reflect
// value to figure out what kind of object we are dealing with and formats it
// appropriately. It is a recursive function, however circular data structures
// are detected and handled properly.
func (d *dumpState) dump(v reflect.Value) {
// Handle invalid reflect values immediately.
kind := v.Kind()
if kind == reflect.Invalid {
d.w.Write(invalidAngleBytes)
return
}
// Handle pointers specially.
if kind == reflect.Ptr {
d.indent()
d.dumpPtr(v)
return
}
// Print type information unless already handled elsewhere.
if !d.ignoreNextType {
d.indent()
d.w.Write(openParenBytes)
d.w.Write([]byte(v.Type().String()))
d.w.Write(closeParenBytes)
d.w.Write(spaceBytes)
}
d.ignoreNextType = false
// Display length and capacity if the built-in len and cap functions
// work with the value's kind and the len/cap itself is non-zero.
valueLen, valueCap := 0, 0
switch v.Kind() {
case reflect.Array, reflect.Slice, reflect.Chan:
valueLen, valueCap = v.Len(), v.Cap()
case reflect.Map, reflect.String:
valueLen = v.Len()
}
if valueLen != 0 || !d.cs.DisableCapacities && valueCap != 0 {
d.w.Write(openParenBytes)
if valueLen != 0 {
d.w.Write(lenEqualsBytes)
printInt(d.w, int64(valueLen), 10)
}
if !d.cs.DisableCapacities && valueCap != 0 {
if valueLen != 0 {
d.w.Write(spaceBytes)
}
d.w.Write(capEqualsBytes)
printInt(d.w, int64(valueCap), 10)
}
d.w.Write(closeParenBytes)
d.w.Write(spaceBytes)
}
// Call Stringer/error interfaces if they exist and the handle methods flag
// is enabled
if !d.cs.DisableMethods {
if (kind != reflect.Invalid) && (kind != reflect.Interface) {
if handled := handleMethods(d.cs, d.w, v); handled {
return
}
}
}
switch kind {
case reflect.Invalid:
// Do nothing. We should never get here since invalid has already
// been handled above.
case reflect.Bool:
printBool(d.w, v.Bool())
case reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Int:
printInt(d.w, v.Int(), 10)
case reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uint:
printUint(d.w, v.Uint(), 10)
case reflect.Float32:
printFloat(d.w, v.Float(), 32)
case reflect.Float64:
printFloat(d.w, v.Float(), 64)
case reflect.Complex64:
printComplex(d.w, v.Complex(), 32)
case reflect.Complex128:
printComplex(d.w, v.Complex(), 64)
case reflect.Slice:
if v.IsNil() {
d.w.Write(nilAngleBytes)
break
}
fallthrough
case reflect.Array:
d.w.Write(openBraceNewlineBytes)
d.depth++
if (d.cs.MaxDepth != 0) && (d.depth > d.cs.MaxDepth) {
d.indent()
d.w.Write(maxNewlineBytes)
} else {
d.dumpSlice(v)
}
d.depth--
d.indent()
d.w.Write(closeBraceBytes)
case reflect.String:
d.w.Write([]byte(strconv.Quote(v.String())))
case reflect.Interface:
// The only time we should get here is for nil interfaces due to
// unpackValue calls.
if v.IsNil() {
d.w.Write(nilAngleBytes)
}
case reflect.Ptr:
// Do nothing. We should never get here since pointers have already
// been handled above.
case reflect.Map:
// nil maps should be indicated as different than empty maps
if v.IsNil() {
d.w.Write(nilAngleBytes)
break
}
d.w.Write(openBraceNewlineBytes)
d.depth++
if (d.cs.MaxDepth != 0) && (d.depth > d.cs.MaxDepth) {
d.indent()
d.w.Write(maxNewlineBytes)
} else {
numEntries := v.Len()
keys := v.MapKeys()
if d.cs.SortKeys {
sortValues(keys, d.cs)
}
for i, key := range keys {
d.dump(d.unpackValue(key))
d.w.Write(colonSpaceBytes)
d.ignoreNextIndent = true
d.dump(d.unpackValue(v.MapIndex(key)))
if i < (numEntries - 1) {
d.w.Write(commaNewlineBytes)
} else {
d.w.Write(newlineBytes)
}
}
}
d.depth--
d.indent()
d.w.Write(closeBraceBytes)
case reflect.Struct:
d.w.Write(openBraceNewlineBytes)
d.depth++
if (d.cs.MaxDepth != 0) && (d.depth > d.cs.MaxDepth) {
d.indent()
d.w.Write(maxNewlineBytes)
} else {
vt := v.Type()
numFields := v.NumField()
for i := 0; i < numFields; i++ {
d.indent()
vtf := vt.Field(i)
d.w.Write([]byte(vtf.Name))
d.w.Write(colonSpaceBytes)
d.ignoreNextIndent = true
d.dump(d.unpackValue(v.Field(i)))
if i < (numFields - 1) {
d.w.Write(commaNewlineBytes)
} else {
d.w.Write(newlineBytes)
}
}
}
d.depth--
d.indent()
d.w.Write(closeBraceBytes)
case reflect.Uintptr:
printHexPtr(d.w, uintptr(v.Uint()))
case reflect.UnsafePointer, reflect.Chan, reflect.Func:
printHexPtr(d.w, v.Pointer())
// There were not any other types at the time this code was written, but
// fall back to letting the default fmt package handle it in case any new
// types are added.
default:
if v.CanInterface() {
fmt.Fprintf(d.w, "%v", v.Interface())
} else {
fmt.Fprintf(d.w, "%v", v.String())
}
}
}
// fdump is a helper function to consolidate the logic from the various public
// methods which take varying writers and config states.
func fdump(cs *ConfigState, w io.Writer, a ...interface{}) {
for _, arg := range a {
if arg == nil {
w.Write(interfaceBytes)
w.Write(spaceBytes)
w.Write(nilAngleBytes)
w.Write(newlineBytes)
continue
}
d := dumpState{w: w, cs: cs}
d.pointers = make(map[uintptr]int)
d.dump(reflect.ValueOf(arg))
d.w.Write(newlineBytes)
}
}
// Fdump formats and displays the passed arguments to io.Writer w. It formats
// exactly the same as Dump.
func Fdump(w io.Writer, a ...interface{}) {
fdump(&Config, w, a...)
}
// Sdump returns a string with the passed arguments formatted exactly the same
// as Dump.
func Sdump(a ...interface{}) string {
var buf bytes.Buffer
fdump(&Config, &buf, a...)
return buf.String()
}
/*
Dump displays the passed parameters to standard out with newlines, customizable
indentation, and additional debug information such as complete types and all
pointer addresses used to indirect to the final value. It provides the
following features over the built-in printing facilities provided by the fmt
package:
* Pointers are dereferenced and followed
* Circular data structures are detected and handled properly
* Custom Stringer/error interfaces are optionally invoked, including
on unexported types
* Custom types which only implement the Stringer/error interfaces via
a pointer receiver are optionally invoked when passing non-pointer
variables
* Byte arrays and slices are dumped like the hexdump -C command which
includes offsets, byte values in hex, and ASCII output
The configuration options are controlled by an exported package global,
spew.Config. See ConfigState for options documentation.
See Fdump if you would prefer dumping to an arbitrary io.Writer or Sdump to
get the formatted result as a string.
*/
func Dump(a ...interface{}) {
fdump(&Config, os.Stdout, a...)
}

View File

@@ -1,419 +0,0 @@
/*
* Copyright (c) 2013-2016 Dave Collins <dave@davec.name>
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
package spew
import (
"bytes"
"fmt"
"reflect"
"strconv"
"strings"
)
// supportedFlags is a list of all the character flags supported by fmt package.
const supportedFlags = "0-+# "
// formatState implements the fmt.Formatter interface and contains information
// about the state of a formatting operation. The NewFormatter function can
// be used to get a new Formatter which can be used directly as arguments
// in standard fmt package printing calls.
type formatState struct {
value interface{}
fs fmt.State
depth int
pointers map[uintptr]int
ignoreNextType bool
cs *ConfigState
}
// buildDefaultFormat recreates the original format string without precision
// and width information to pass in to fmt.Sprintf in the case of an
// unrecognized type. Unless new types are added to the language, this
// function won't ever be called.
func (f *formatState) buildDefaultFormat() (format string) {
buf := bytes.NewBuffer(percentBytes)
for _, flag := range supportedFlags {
if f.fs.Flag(int(flag)) {
buf.WriteRune(flag)
}
}
buf.WriteRune('v')
format = buf.String()
return format
}
// constructOrigFormat recreates the original format string including precision
// and width information to pass along to the standard fmt package. This allows
// automatic deferral of all format strings this package doesn't support.
func (f *formatState) constructOrigFormat(verb rune) (format string) {
buf := bytes.NewBuffer(percentBytes)
for _, flag := range supportedFlags {
if f.fs.Flag(int(flag)) {
buf.WriteRune(flag)
}
}
if width, ok := f.fs.Width(); ok {
buf.WriteString(strconv.Itoa(width))
}
if precision, ok := f.fs.Precision(); ok {
buf.Write(precisionBytes)
buf.WriteString(strconv.Itoa(precision))
}
buf.WriteRune(verb)
format = buf.String()
return format
}
// unpackValue returns values inside of non-nil interfaces when possible and
// ensures that types for values which have been unpacked from an interface
// are displayed when the show types flag is also set.
// This is useful for data types like structs, arrays, slices, and maps which
// can contain varying types packed inside an interface.
func (f *formatState) unpackValue(v reflect.Value) reflect.Value {
if v.Kind() == reflect.Interface {
f.ignoreNextType = false
if !v.IsNil() {
v = v.Elem()
}
}
return v
}
// formatPtr handles formatting of pointers by indirecting them as necessary.
func (f *formatState) formatPtr(v reflect.Value) {
// Display nil if top level pointer is nil.
showTypes := f.fs.Flag('#')
if v.IsNil() && (!showTypes || f.ignoreNextType) {
f.fs.Write(nilAngleBytes)
return
}
// Remove pointers at or below the current depth from map used to detect
// circular refs.
for k, depth := range f.pointers {
if depth >= f.depth {
delete(f.pointers, k)
}
}
// Keep list of all dereferenced pointers to possibly show later.
pointerChain := make([]uintptr, 0)
// Figure out how many levels of indirection there are by derferencing
// pointers and unpacking interfaces down the chain while detecting circular
// references.
nilFound := false
cycleFound := false
indirects := 0
ve := v
for ve.Kind() == reflect.Ptr {
if ve.IsNil() {
nilFound = true
break
}
indirects++
addr := ve.Pointer()
pointerChain = append(pointerChain, addr)
if pd, ok := f.pointers[addr]; ok && pd < f.depth {
cycleFound = true
indirects--
break
}
f.pointers[addr] = f.depth
ve = ve.Elem()
if ve.Kind() == reflect.Interface {
if ve.IsNil() {
nilFound = true
break
}
ve = ve.Elem()
}
}
// Display type or indirection level depending on flags.
if showTypes && !f.ignoreNextType {
f.fs.Write(openParenBytes)
f.fs.Write(bytes.Repeat(asteriskBytes, indirects))
f.fs.Write([]byte(ve.Type().String()))
f.fs.Write(closeParenBytes)
} else {
if nilFound || cycleFound {
indirects += strings.Count(ve.Type().String(), "*")
}
f.fs.Write(openAngleBytes)
f.fs.Write([]byte(strings.Repeat("*", indirects)))
f.fs.Write(closeAngleBytes)
}
// Display pointer information depending on flags.
if f.fs.Flag('+') && (len(pointerChain) > 0) {
f.fs.Write(openParenBytes)
for i, addr := range pointerChain {
if i > 0 {
f.fs.Write(pointerChainBytes)
}
printHexPtr(f.fs, addr)
}
f.fs.Write(closeParenBytes)
}
// Display dereferenced value.
switch {
case nilFound:
f.fs.Write(nilAngleBytes)
case cycleFound:
f.fs.Write(circularShortBytes)
default:
f.ignoreNextType = true
f.format(ve)
}
}
// format is the main workhorse for providing the Formatter interface. It
// uses the passed reflect value to figure out what kind of object we are
// dealing with and formats it appropriately. It is a recursive function,
// however circular data structures are detected and handled properly.
func (f *formatState) format(v reflect.Value) {
// Handle invalid reflect values immediately.
kind := v.Kind()
if kind == reflect.Invalid {
f.fs.Write(invalidAngleBytes)
return
}
// Handle pointers specially.
if kind == reflect.Ptr {
f.formatPtr(v)
return
}
// Print type information unless already handled elsewhere.
if !f.ignoreNextType && f.fs.Flag('#') {
f.fs.Write(openParenBytes)
f.fs.Write([]byte(v.Type().String()))
f.fs.Write(closeParenBytes)
}
f.ignoreNextType = false
// Call Stringer/error interfaces if they exist and the handle methods
// flag is enabled.
if !f.cs.DisableMethods {
if (kind != reflect.Invalid) && (kind != reflect.Interface) {
if handled := handleMethods(f.cs, f.fs, v); handled {
return
}
}
}
switch kind {
case reflect.Invalid:
// Do nothing. We should never get here since invalid has already
// been handled above.
case reflect.Bool:
printBool(f.fs, v.Bool())
case reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Int:
printInt(f.fs, v.Int(), 10)
case reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uint:
printUint(f.fs, v.Uint(), 10)
case reflect.Float32:
printFloat(f.fs, v.Float(), 32)
case reflect.Float64:
printFloat(f.fs, v.Float(), 64)
case reflect.Complex64:
printComplex(f.fs, v.Complex(), 32)
case reflect.Complex128:
printComplex(f.fs, v.Complex(), 64)
case reflect.Slice:
if v.IsNil() {
f.fs.Write(nilAngleBytes)
break
}
fallthrough
case reflect.Array:
f.fs.Write(openBracketBytes)
f.depth++
if (f.cs.MaxDepth != 0) && (f.depth > f.cs.MaxDepth) {
f.fs.Write(maxShortBytes)
} else {
numEntries := v.Len()
for i := 0; i < numEntries; i++ {
if i > 0 {
f.fs.Write(spaceBytes)
}
f.ignoreNextType = true
f.format(f.unpackValue(v.Index(i)))
}
}
f.depth--
f.fs.Write(closeBracketBytes)
case reflect.String:
f.fs.Write([]byte(v.String()))
case reflect.Interface:
// The only time we should get here is for nil interfaces due to
// unpackValue calls.
if v.IsNil() {
f.fs.Write(nilAngleBytes)
}
case reflect.Ptr:
// Do nothing. We should never get here since pointers have already
// been handled above.
case reflect.Map:
// nil maps should be indicated as different than empty maps
if v.IsNil() {
f.fs.Write(nilAngleBytes)
break
}
f.fs.Write(openMapBytes)
f.depth++
if (f.cs.MaxDepth != 0) && (f.depth > f.cs.MaxDepth) {
f.fs.Write(maxShortBytes)
} else {
keys := v.MapKeys()
if f.cs.SortKeys {
sortValues(keys, f.cs)
}
for i, key := range keys {
if i > 0 {
f.fs.Write(spaceBytes)
}
f.ignoreNextType = true
f.format(f.unpackValue(key))
f.fs.Write(colonBytes)
f.ignoreNextType = true
f.format(f.unpackValue(v.MapIndex(key)))
}
}
f.depth--
f.fs.Write(closeMapBytes)
case reflect.Struct:
numFields := v.NumField()
f.fs.Write(openBraceBytes)
f.depth++
if (f.cs.MaxDepth != 0) && (f.depth > f.cs.MaxDepth) {
f.fs.Write(maxShortBytes)
} else {
vt := v.Type()
for i := 0; i < numFields; i++ {
if i > 0 {
f.fs.Write(spaceBytes)
}
vtf := vt.Field(i)
if f.fs.Flag('+') || f.fs.Flag('#') {
f.fs.Write([]byte(vtf.Name))
f.fs.Write(colonBytes)
}
f.format(f.unpackValue(v.Field(i)))
}
}
f.depth--
f.fs.Write(closeBraceBytes)
case reflect.Uintptr:
printHexPtr(f.fs, uintptr(v.Uint()))
case reflect.UnsafePointer, reflect.Chan, reflect.Func:
printHexPtr(f.fs, v.Pointer())
// There were not any other types at the time this code was written, but
// fall back to letting the default fmt package handle it if any get added.
default:
format := f.buildDefaultFormat()
if v.CanInterface() {
fmt.Fprintf(f.fs, format, v.Interface())
} else {
fmt.Fprintf(f.fs, format, v.String())
}
}
}
// Format satisfies the fmt.Formatter interface. See NewFormatter for usage
// details.
func (f *formatState) Format(fs fmt.State, verb rune) {
f.fs = fs
// Use standard formatting for verbs that are not v.
if verb != 'v' {
format := f.constructOrigFormat(verb)
fmt.Fprintf(fs, format, f.value)
return
}
if f.value == nil {
if fs.Flag('#') {
fs.Write(interfaceBytes)
}
fs.Write(nilAngleBytes)
return
}
f.format(reflect.ValueOf(f.value))
}
// newFormatter is a helper function to consolidate the logic from the various
// public methods which take varying config states.
func newFormatter(cs *ConfigState, v interface{}) fmt.Formatter {
fs := &formatState{value: v, cs: cs}
fs.pointers = make(map[uintptr]int)
return fs
}
/*
NewFormatter returns a custom formatter that satisfies the fmt.Formatter
interface. As a result, it integrates cleanly with standard fmt package
printing functions. The formatter is useful for inline printing of smaller data
types similar to the standard %v format specifier.
The custom formatter only responds to the %v (most compact), %+v (adds pointer
addresses), %#v (adds types), or %#+v (adds types and pointer addresses) verb
combinations. Any other verbs such as %x and %q will be sent to the the
standard fmt package for formatting. In addition, the custom formatter ignores
the width and precision arguments (however they will still work on the format
specifiers not handled by the custom formatter).
Typically this function shouldn't be called directly. It is much easier to make
use of the custom formatter by calling one of the convenience functions such as
Printf, Println, or Fprintf.
*/
func NewFormatter(v interface{}) fmt.Formatter {
return newFormatter(&Config, v)
}

View File

@@ -1,148 +0,0 @@
/*
* Copyright (c) 2013-2016 Dave Collins <dave@davec.name>
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
package spew
import (
"fmt"
"io"
)
// Errorf is a wrapper for fmt.Errorf that treats each argument as if it were
// passed with a default Formatter interface returned by NewFormatter. It
// returns the formatted string as a value that satisfies error. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Errorf(format, spew.NewFormatter(a), spew.NewFormatter(b))
func Errorf(format string, a ...interface{}) (err error) {
return fmt.Errorf(format, convertArgs(a)...)
}
// Fprint is a wrapper for fmt.Fprint that treats each argument as if it were
// passed with a default Formatter interface returned by NewFormatter. It
// returns the number of bytes written and any write error encountered. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Fprint(w, spew.NewFormatter(a), spew.NewFormatter(b))
func Fprint(w io.Writer, a ...interface{}) (n int, err error) {
return fmt.Fprint(w, convertArgs(a)...)
}
// Fprintf is a wrapper for fmt.Fprintf that treats each argument as if it were
// passed with a default Formatter interface returned by NewFormatter. It
// returns the number of bytes written and any write error encountered. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Fprintf(w, format, spew.NewFormatter(a), spew.NewFormatter(b))
func Fprintf(w io.Writer, format string, a ...interface{}) (n int, err error) {
return fmt.Fprintf(w, format, convertArgs(a)...)
}
// Fprintln is a wrapper for fmt.Fprintln that treats each argument as if it
// passed with a default Formatter interface returned by NewFormatter. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Fprintln(w, spew.NewFormatter(a), spew.NewFormatter(b))
func Fprintln(w io.Writer, a ...interface{}) (n int, err error) {
return fmt.Fprintln(w, convertArgs(a)...)
}
// Print is a wrapper for fmt.Print that treats each argument as if it were
// passed with a default Formatter interface returned by NewFormatter. It
// returns the number of bytes written and any write error encountered. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Print(spew.NewFormatter(a), spew.NewFormatter(b))
func Print(a ...interface{}) (n int, err error) {
return fmt.Print(convertArgs(a)...)
}
// Printf is a wrapper for fmt.Printf that treats each argument as if it were
// passed with a default Formatter interface returned by NewFormatter. It
// returns the number of bytes written and any write error encountered. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Printf(format, spew.NewFormatter(a), spew.NewFormatter(b))
func Printf(format string, a ...interface{}) (n int, err error) {
return fmt.Printf(format, convertArgs(a)...)
}
// Println is a wrapper for fmt.Println that treats each argument as if it were
// passed with a default Formatter interface returned by NewFormatter. It
// returns the number of bytes written and any write error encountered. See
// NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Println(spew.NewFormatter(a), spew.NewFormatter(b))
func Println(a ...interface{}) (n int, err error) {
return fmt.Println(convertArgs(a)...)
}
// Sprint is a wrapper for fmt.Sprint that treats each argument as if it were
// passed with a default Formatter interface returned by NewFormatter. It
// returns the resulting string. See NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Sprint(spew.NewFormatter(a), spew.NewFormatter(b))
func Sprint(a ...interface{}) string {
return fmt.Sprint(convertArgs(a)...)
}
// Sprintf is a wrapper for fmt.Sprintf that treats each argument as if it were
// passed with a default Formatter interface returned by NewFormatter. It
// returns the resulting string. See NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Sprintf(format, spew.NewFormatter(a), spew.NewFormatter(b))
func Sprintf(format string, a ...interface{}) string {
return fmt.Sprintf(format, convertArgs(a)...)
}
// Sprintln is a wrapper for fmt.Sprintln that treats each argument as if it
// were passed with a default Formatter interface returned by NewFormatter. It
// returns the resulting string. See NewFormatter for formatting details.
//
// This function is shorthand for the following syntax:
//
// fmt.Sprintln(spew.NewFormatter(a), spew.NewFormatter(b))
func Sprintln(a ...interface{}) string {
return fmt.Sprintln(convertArgs(a)...)
}
// convertArgs accepts a slice of arguments and returns a slice of the same
// length with each argument converted to a default spew Formatter interface.
func convertArgs(args []interface{}) (formatters []interface{}) {
formatters = make([]interface{}, len(args))
for index, arg := range args {
formatters[index] = NewFormatter(arg)
}
return formatters
}

View File

@@ -1,71 +0,0 @@
# Compiled Object files, Static and Dynamic libs (Shared Objects)
*.o
*.a
*.so
# Folders
_obj
_test
# Architecture specific extensions/prefixes
*.[568vq]
[568vq].out
*.cgo1.go
*.cgo2.c
_cgo_defun.c
_cgo_gotypes.go
_cgo_export.*
_testmain.go
*.exe
restful.html
*.out
tmp.prof
go-restful.test
examples/restful-basic-authentication
examples/restful-encoding-filter
examples/restful-filters
examples/restful-hello-world
examples/restful-resource-functions
examples/restful-serve-static
examples/restful-user-service
*.DS_Store
examples/restful-user-resource
examples/restful-multi-containers
examples/restful-form-handling
examples/restful-CORS-filter
examples/restful-options-filter
examples/restful-curly-router
examples/restful-cpuprofiler-service
examples/restful-pre-post-filters
curly.prof
examples/restful-NCSA-logging
examples/restful-html-template
s.html
restful-path-tail
.idea

View File

@@ -1 +0,0 @@
ignore

View File

@@ -1,13 +0,0 @@
language: go
go:
- 1.x
before_install:
- go test -v
script:
- go test -race -coverprofile=coverage.txt -covermode=atomic
after_success:
- bash <(curl -s https://codecov.io/bash)

View File

@@ -1,396 +0,0 @@
# Change history of go-restful
## [v3.11.0] - 2023-08-19
- restored behavior as <= v3.9.0 with option to change path strategy using TrimRightSlashEnabled.
## [v3.10.2] - 2023-03-09 - DO NOT USE
- introduced MergePathStrategy to be able to revert behaviour of path concatenation to 3.9.0
see comment in Readme how to customize this behaviour.
## [v3.10.1] - 2022-11-19 - DO NOT USE
- fix broken 3.10.0 by using path package for joining paths
## [v3.10.0] - 2022-10-11 - BROKEN
- changed tokenizer to match std route match behavior; do not trimright the path (#511)
- Add MIME_ZIP (#512)
- Add MIME_ZIP and HEADER_ContentDisposition (#513)
- Changed how to get query parameter issue #510
## [v3.9.0] - 2022-07-21
- add support for http.Handler implementations to work as FilterFunction, issue #504 (thanks to https://github.com/ggicci)
## [v3.8.0] - 2022-06-06
- use exact matching of allowed domain entries, issue #489 (#493)
- this changes fixes [security] Authorization Bypass Through User-Controlled Key
by changing the behaviour of the AllowedDomains setting in the CORS filter.
To support the previous behaviour, the CORS filter type now has a AllowedDomainFunc
callback mechanism which is called when a simple domain match fails.
- add test and fix for POST without body and Content-type, issue #492 (#496)
- [Minor] Bad practice to have a mix of Receiver types. (#491)
## [v3.7.2] - 2021-11-24
- restored FilterChain (#482 by SVilgelm)
## [v3.7.1] - 2021-10-04
- fix problem with contentEncodingEnabled setting (#479)
## [v3.7.0] - 2021-09-24
- feat(parameter): adds additional openapi mappings (#478)
## [v3.6.0] - 2021-09-18
- add support for vendor extensions (#477 thx erraggy)
## [v3.5.2] - 2021-07-14
- fix removing absent route from webservice (#472)
## [v3.5.1] - 2021-04-12
- fix handling no match access selected path
- remove obsolete field
## [v3.5.0] - 2021-04-10
- add check for wildcard (#463) in CORS
- add access to Route from Request, issue #459 (#462)
## [v3.4.0] - 2020-11-10
- Added OPTIONS to WebService
## [v3.3.2] - 2020-01-23
- Fixed duplicate compression in dispatch. #449
## [v3.3.1] - 2020-08-31
- Added check on writer to prevent compression of response twice. #447
## [v3.3.0] - 2020-08-19
- Enable content encoding on Handle and ServeHTTP (#446)
- List available representations in 406 body (#437)
- Convert to string using rune() (#443)
## [v3.2.0] - 2020-06-21
- 405 Method Not Allowed must have Allow header (#436) (thx Bracken <abdawson@gmail.com>)
- add field allowedMethodsWithoutContentType (#424)
## [v3.1.0]
- support describing response headers (#426)
- fix openapi examples (#425)
v3.0.0
- fix: use request/response resulting from filter chain
- add Go module
Module consumer should use github.com/emicklei/go-restful/v3 as import path
v2.10.0
- support for Custom Verbs (thanks Vinci Xu <277040271@qq.com>)
- fixed static example (thanks Arthur <yang_yapo@126.com>)
- simplify code (thanks Christian Muehlhaeuser <muesli@gmail.com>)
- added JWT HMAC with SHA-512 authentication code example (thanks Amim Knabben <amim.knabben@gmail.com>)
v2.9.6
- small optimization in filter code
v2.11.1
- fix WriteError return value (#415)
v2.11.0
- allow prefix and suffix in path variable expression (#414)
v2.9.6
- support google custome verb (#413)
v2.9.5
- fix panic in Response.WriteError if err == nil
v2.9.4
- fix issue #400 , parsing mime type quality
- Route Builder added option for contentEncodingEnabled (#398)
v2.9.3
- Avoid return of 415 Unsupported Media Type when request body is empty (#396)
v2.9.2
- Reduce allocations in per-request methods to improve performance (#395)
v2.9.1
- Fix issue with default responses and invalid status code 0. (#393)
v2.9.0
- add per Route content encoding setting (overrides container setting)
v2.8.0
- add Request.QueryParameters()
- add json-iterator (via build tag)
- disable vgo module (until log is moved)
v2.7.1
- add vgo module
v2.6.1
- add JSONNewDecoderFunc to allow custom JSON Decoder usage (go 1.10+)
v2.6.0
- Make JSR 311 routing and path param processing consistent
- Adding description to RouteBuilder.Reads()
- Update example for Swagger12 and OpenAPI
2017-09-13
- added route condition functions using `.If(func)` in route building.
2017-02-16
- solved issue #304, make operation names unique
2017-01-30
[IMPORTANT] For swagger users, change your import statement to:
swagger "github.com/emicklei/go-restful-swagger12"
- moved swagger 1.2 code to go-restful-swagger12
- created TAG 2.0.0
2017-01-27
- remove defer request body close
- expose Dispatch for testing filters and Routefunctions
- swagger response model cannot be array
- created TAG 1.0.0
2016-12-22
- (API change) Remove code related to caching request content. Removes SetCacheReadEntity(doCache bool)
2016-11-26
- Default change! now use CurlyRouter (was RouterJSR311)
- Default change! no more caching of request content
- Default change! do not recover from panics
2016-09-22
- fix the DefaultRequestContentType feature
2016-02-14
- take the qualify factor of the Accept header mediatype into account when deciding the contentype of the response
- add constructors for custom entity accessors for xml and json
2015-09-27
- rename new WriteStatusAnd... to WriteHeaderAnd... for consistency
2015-09-25
- fixed problem with changing Header after WriteHeader (issue 235)
2015-09-14
- changed behavior of WriteHeader (immediate write) and WriteEntity (no status write)
- added support for custom EntityReaderWriters.
2015-08-06
- add support for reading entities from compressed request content
- use sync.Pool for compressors of http response and request body
- add Description to Parameter for documentation in Swagger UI
2015-03-20
- add configurable logging
2015-03-18
- if not specified, the Operation is derived from the Route function
2015-03-17
- expose Parameter creation functions
- make trace logger an interface
- fix OPTIONSFilter
- customize rendering of ServiceError
- JSR311 router now handles wildcards
- add Notes to Route
2014-11-27
- (api add) PrettyPrint per response. (as proposed in #167)
2014-11-12
- (api add) ApiVersion(.) for documentation in Swagger UI
2014-11-10
- (api change) struct fields tagged with "description" show up in Swagger UI
2014-10-31
- (api change) ReturnsError -> Returns
- (api add) RouteBuilder.Do(aBuilder) for DRY use of RouteBuilder
- fix swagger nested structs
- sort Swagger response messages by code
2014-10-23
- (api add) ReturnsError allows you to document Http codes in swagger
- fixed problem with greedy CurlyRouter
- (api add) Access-Control-Max-Age in CORS
- add tracing functionality (injectable) for debugging purposes
- support JSON parse 64bit int
- fix empty parameters for swagger
- WebServicesUrl is now optional for swagger
- fixed duplicate AccessControlAllowOrigin in CORS
- (api change) expose ServeMux in container
- (api add) added AllowedDomains in CORS
- (api add) ParameterNamed for detailed documentation
2014-04-16
- (api add) expose constructor of Request for testing.
2014-06-27
- (api add) ParameterNamed gives access to a Parameter definition and its data (for further specification).
- (api add) SetCacheReadEntity allow scontrol over whether or not the request body is being cached (default true for compatibility reasons).
2014-07-03
- (api add) CORS can be configured with a list of allowed domains
2014-03-12
- (api add) Route path parameters can use wildcard or regular expressions. (requires CurlyRouter)
2014-02-26
- (api add) Request now provides information about the matched Route, see method SelectedRoutePath
2014-02-17
- (api change) renamed parameter constants (go-lint checks)
2014-01-10
- (api add) support for CloseNotify, see http://golang.org/pkg/net/http/#CloseNotifier
2014-01-07
- (api change) Write* methods in Response now return the error or nil.
- added example of serving HTML from a Go template.
- fixed comparing Allowed headers in CORS (is now case-insensitive)
2013-11-13
- (api add) Response knows how many bytes are written to the response body.
2013-10-29
- (api add) RecoverHandler(handler RecoverHandleFunction) to change how panic recovery is handled. Default behavior is to log and return a stacktrace. This may be a security issue as it exposes sourcecode information.
2013-10-04
- (api add) Response knows what HTTP status has been written
- (api add) Request can have attributes (map of string->interface, also called request-scoped variables
2013-09-12
- (api change) Router interface simplified
- Implemented CurlyRouter, a Router that does not use|allow regular expressions in paths
2013-08-05
- add OPTIONS support
- add CORS support
2013-08-27
- fixed some reported issues (see github)
- (api change) deprecated use of WriteError; use WriteErrorString instead
2014-04-15
- (fix) v1.0.1 tag: fix Issue 111: WriteErrorString
2013-08-08
- (api add) Added implementation Container: a WebServices collection with its own http.ServeMux allowing multiple endpoints per program. Existing uses of go-restful will register their services to the DefaultContainer.
- (api add) the swagger package has be extended to have a UI per container.
- if panic is detected then a small stack trace is printed (thanks to runner-mei)
- (api add) WriteErrorString to Response
Important API changes:
- (api remove) package variable DoNotRecover no longer works ; use restful.DefaultContainer.DoNotRecover(true) instead.
- (api remove) package variable EnableContentEncoding no longer works ; use restful.DefaultContainer.EnableContentEncoding(true) instead.
2013-07-06
- (api add) Added support for response encoding (gzip and deflate(zlib)). This feature is disabled on default (for backwards compatibility). Use restful.EnableContentEncoding = true in your initialization to enable this feature.
2013-06-19
- (improve) DoNotRecover option, moved request body closer, improved ReadEntity
2013-06-03
- (api change) removed Dispatcher interface, hide PathExpression
- changed receiver names of type functions to be more idiomatic Go
2013-06-02
- (optimize) Cache the RegExp compilation of Paths.
2013-05-22
- (api add) Added support for request/response filter functions
2013-05-18
- (api add) Added feature to change the default Http Request Dispatch function (travis cline)
- (api change) Moved Swagger Webservice to swagger package (see example restful-user)
[2012-11-14 .. 2013-05-18>
- See https://github.com/emicklei/go-restful/commits
2012-11-14
- Initial commit

View File

@@ -1,22 +0,0 @@
Copyright (c) 2012,2013 Ernest Micklei
MIT License
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

View File

@@ -1,8 +0,0 @@
all: test
test:
go vet .
go test -cover -v .
ex:
find ./examples -type f -name "*.go" | xargs -I {} go build -o /tmp/ignore {}

View File

@@ -1,112 +0,0 @@
go-restful
==========
package for building REST-style Web Services using Google Go
[![Build Status](https://travis-ci.org/emicklei/go-restful.png)](https://travis-ci.org/emicklei/go-restful)
[![Go Report Card](https://goreportcard.com/badge/github.com/emicklei/go-restful)](https://goreportcard.com/report/github.com/emicklei/go-restful)
[![GoDoc](https://godoc.org/github.com/emicklei/go-restful?status.svg)](https://pkg.go.dev/github.com/emicklei/go-restful)
[![codecov](https://codecov.io/gh/emicklei/go-restful/branch/master/graph/badge.svg)](https://codecov.io/gh/emicklei/go-restful)
- [Code examples use v3](https://github.com/emicklei/go-restful/tree/v3/examples)
REST asks developers to use HTTP methods explicitly and in a way that's consistent with the protocol definition. This basic REST design principle establishes a one-to-one mapping between create, read, update, and delete (CRUD) operations and HTTP methods. According to this mapping:
- GET = Retrieve a representation of a resource
- POST = Create if you are sending content to the server to create a subordinate of the specified resource collection, using some server-side algorithm.
- PUT = Create if you are sending the full content of the specified resource (URI).
- PUT = Update if you are updating the full content of the specified resource.
- DELETE = Delete if you are requesting the server to delete the resource
- PATCH = Update partial content of a resource
- OPTIONS = Get information about the communication options for the request URI
### Usage
#### Without Go Modules
All versions up to `v2.*.*` (on the master) are not supporting Go modules.
```
import (
restful "github.com/emicklei/go-restful"
)
```
#### Using Go Modules
As of version `v3.0.0` (on the v3 branch), this package supports Go modules.
```
import (
restful "github.com/emicklei/go-restful/v3"
)
```
### Example
```Go
ws := new(restful.WebService)
ws.
Path("/users").
Consumes(restful.MIME_XML, restful.MIME_JSON).
Produces(restful.MIME_JSON, restful.MIME_XML)
ws.Route(ws.GET("/{user-id}").To(u.findUser).
Doc("get a user").
Param(ws.PathParameter("user-id", "identifier of the user").DataType("string")).
Writes(User{}))
...
func (u UserResource) findUser(request *restful.Request, response *restful.Response) {
id := request.PathParameter("user-id")
...
}
```
[Full API of a UserResource](https://github.com/emicklei/go-restful/blob/v3/examples/user-resource/restful-user-resource.go)
### Features
- Routes for request &#8594; function mapping with path parameter (e.g. {id} but also prefix_{var} and {var}_suffix) support
- Configurable router:
- (default) Fast routing algorithm that allows static elements, [google custom method](https://cloud.google.com/apis/design/custom_methods), regular expressions and dynamic parameters in the URL path (e.g. /resource/name:customVerb, /meetings/{id} or /static/{subpath:*})
- Routing algorithm after [JSR311](http://jsr311.java.net/nonav/releases/1.1/spec/spec.html) that is implemented using (but does **not** accept) regular expressions
- Request API for reading structs from JSON/XML and accessing parameters (path,query,header)
- Response API for writing structs to JSON/XML and setting headers
- Customizable encoding using EntityReaderWriter registration
- Filters for intercepting the request &#8594; response flow on Service or Route level
- Request-scoped variables using attributes
- Containers for WebServices on different HTTP endpoints
- Content encoding (gzip,deflate) of request and response payloads
- Automatic responses on OPTIONS (using a filter)
- Automatic CORS request handling (using a filter)
- API declaration for Swagger UI ([go-restful-openapi](https://github.com/emicklei/go-restful-openapi))
- Panic recovery to produce HTTP 500, customizable using RecoverHandler(...)
- Route errors produce HTTP 404/405/406/415 errors, customizable using ServiceErrorHandler(...)
- Configurable (trace) logging
- Customizable gzip/deflate readers and writers using CompressorProvider registration
- Inject your own http.Handler using the `HttpMiddlewareHandlerToFilter` function
## How to customize
There are several hooks to customize the behavior of the go-restful package.
- Router algorithm
- Panic recovery
- JSON decoder
- Trace logging
- Compression
- Encoders for other serializers
- Use [jsoniter](https://github.com/json-iterator/go) by building this package using a build tag, e.g. `go build -tags=jsoniter .`
- Use the package variable `TrimRightSlashEnabled` (default true) to control the behavior of matching routes that end with a slash `/`
## Resources
- [Example programs](./examples)
- [Example posted on blog](http://ernestmicklei.com/2012/11/go-restful-first-working-example/)
- [Design explained on blog](http://ernestmicklei.com/2012/11/go-restful-api-design/)
- [sourcegraph](https://sourcegraph.com/github.com/emicklei/go-restful)
- [showcase: Zazkia - tcp proxy for testing resiliency](https://github.com/emicklei/zazkia)
- [showcase: Mora - MongoDB REST Api server](https://github.com/emicklei/mora)
Type ```git shortlog -s``` for a full list of contributors.
© 2012 - 2023, http://ernestmicklei.com. MIT License. Contributions are welcome.

View File

@@ -1,13 +0,0 @@
# Security Policy
## Supported Versions
| Version | Supported |
| ------- | ------------------ |
| v3.7.x | :white_check_mark: |
| < v3.0.1 | :x: |
## Reporting a Vulnerability
Create an Issue and put the label `[security]` in the title of the issue.
Valid reported security issues are expected to be solved within a week.

View File

@@ -1 +0,0 @@
{"SkipDirs": ["examples"]}

View File

@@ -1,10 +0,0 @@
#go test -run=none -file bench_test.go -test.bench . -cpuprofile=bench_test.out
go test -c
./go-restful.test -test.run=none -test.cpuprofile=tmp.prof -test.bench=BenchmarkMany
./go-restful.test -test.run=none -test.cpuprofile=curly.prof -test.bench=BenchmarkManyCurly
#go tool pprof go-restful.test tmp.prof
go tool pprof go-restful.test curly.prof

View File

@@ -1,127 +0,0 @@
package restful
// Copyright 2013 Ernest Micklei. All rights reserved.
// Use of this source code is governed by a license
// that can be found in the LICENSE file.
import (
"bufio"
"compress/gzip"
"compress/zlib"
"errors"
"io"
"net"
"net/http"
"strings"
)
// OBSOLETE : use restful.DefaultContainer.EnableContentEncoding(true) to change this setting.
var EnableContentEncoding = false
// CompressingResponseWriter is a http.ResponseWriter that can perform content encoding (gzip and zlib)
type CompressingResponseWriter struct {
writer http.ResponseWriter
compressor io.WriteCloser
encoding string
}
// Header is part of http.ResponseWriter interface
func (c *CompressingResponseWriter) Header() http.Header {
return c.writer.Header()
}
// WriteHeader is part of http.ResponseWriter interface
func (c *CompressingResponseWriter) WriteHeader(status int) {
c.writer.WriteHeader(status)
}
// Write is part of http.ResponseWriter interface
// It is passed through the compressor
func (c *CompressingResponseWriter) Write(bytes []byte) (int, error) {
if c.isCompressorClosed() {
return -1, errors.New("Compressing error: tried to write data using closed compressor")
}
return c.compressor.Write(bytes)
}
// CloseNotify is part of http.CloseNotifier interface
func (c *CompressingResponseWriter) CloseNotify() <-chan bool {
return c.writer.(http.CloseNotifier).CloseNotify()
}
// Close the underlying compressor
func (c *CompressingResponseWriter) Close() error {
if c.isCompressorClosed() {
return errors.New("Compressing error: tried to close already closed compressor")
}
c.compressor.Close()
if ENCODING_GZIP == c.encoding {
currentCompressorProvider.ReleaseGzipWriter(c.compressor.(*gzip.Writer))
}
if ENCODING_DEFLATE == c.encoding {
currentCompressorProvider.ReleaseZlibWriter(c.compressor.(*zlib.Writer))
}
// gc hint needed?
c.compressor = nil
return nil
}
func (c *CompressingResponseWriter) isCompressorClosed() bool {
return nil == c.compressor
}
// Hijack implements the Hijacker interface
// This is especially useful when combining Container.EnabledContentEncoding
// in combination with websockets (for instance gorilla/websocket)
func (c *CompressingResponseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
hijacker, ok := c.writer.(http.Hijacker)
if !ok {
return nil, nil, errors.New("ResponseWriter doesn't support Hijacker interface")
}
return hijacker.Hijack()
}
// WantsCompressedResponse reads the Accept-Encoding header to see if and which encoding is requested.
// It also inspects the httpWriter whether its content-encoding is already set (non-empty).
func wantsCompressedResponse(httpRequest *http.Request, httpWriter http.ResponseWriter) (bool, string) {
if contentEncoding := httpWriter.Header().Get(HEADER_ContentEncoding); contentEncoding != "" {
return false, ""
}
header := httpRequest.Header.Get(HEADER_AcceptEncoding)
gi := strings.Index(header, ENCODING_GZIP)
zi := strings.Index(header, ENCODING_DEFLATE)
// use in order of appearance
if gi == -1 {
return zi != -1, ENCODING_DEFLATE
} else if zi == -1 {
return gi != -1, ENCODING_GZIP
} else {
if gi < zi {
return true, ENCODING_GZIP
}
return true, ENCODING_DEFLATE
}
}
// NewCompressingResponseWriter create a CompressingResponseWriter for a known encoding = {gzip,deflate}
func NewCompressingResponseWriter(httpWriter http.ResponseWriter, encoding string) (*CompressingResponseWriter, error) {
httpWriter.Header().Set(HEADER_ContentEncoding, encoding)
c := new(CompressingResponseWriter)
c.writer = httpWriter
var err error
if ENCODING_GZIP == encoding {
w := currentCompressorProvider.AcquireGzipWriter()
w.Reset(httpWriter)
c.compressor = w
c.encoding = ENCODING_GZIP
} else if ENCODING_DEFLATE == encoding {
w := currentCompressorProvider.AcquireZlibWriter()
w.Reset(httpWriter)
c.compressor = w
c.encoding = ENCODING_DEFLATE
} else {
return nil, errors.New("Unknown encoding:" + encoding)
}
return c, err
}

View File

@@ -1,103 +0,0 @@
package restful
// Copyright 2015 Ernest Micklei. All rights reserved.
// Use of this source code is governed by a license
// that can be found in the LICENSE file.
import (
"compress/gzip"
"compress/zlib"
)
// BoundedCachedCompressors is a CompressorProvider that uses a cache with a fixed amount
// of writers and readers (resources).
// If a new resource is acquired and all are in use, it will return a new unmanaged resource.
type BoundedCachedCompressors struct {
gzipWriters chan *gzip.Writer
gzipReaders chan *gzip.Reader
zlibWriters chan *zlib.Writer
writersCapacity int
readersCapacity int
}
// NewBoundedCachedCompressors returns a new, with filled cache, BoundedCachedCompressors.
func NewBoundedCachedCompressors(writersCapacity, readersCapacity int) *BoundedCachedCompressors {
b := &BoundedCachedCompressors{
gzipWriters: make(chan *gzip.Writer, writersCapacity),
gzipReaders: make(chan *gzip.Reader, readersCapacity),
zlibWriters: make(chan *zlib.Writer, writersCapacity),
writersCapacity: writersCapacity,
readersCapacity: readersCapacity,
}
for ix := 0; ix < writersCapacity; ix++ {
b.gzipWriters <- newGzipWriter()
b.zlibWriters <- newZlibWriter()
}
for ix := 0; ix < readersCapacity; ix++ {
b.gzipReaders <- newGzipReader()
}
return b
}
// AcquireGzipWriter returns an resettable *gzip.Writer. Needs to be released.
func (b *BoundedCachedCompressors) AcquireGzipWriter() *gzip.Writer {
var writer *gzip.Writer
select {
case writer, _ = <-b.gzipWriters:
default:
// return a new unmanaged one
writer = newGzipWriter()
}
return writer
}
// ReleaseGzipWriter accepts a writer (does not have to be one that was cached)
// only when the cache has room for it. It will ignore it otherwise.
func (b *BoundedCachedCompressors) ReleaseGzipWriter(w *gzip.Writer) {
// forget the unmanaged ones
if len(b.gzipWriters) < b.writersCapacity {
b.gzipWriters <- w
}
}
// AcquireGzipReader returns a *gzip.Reader. Needs to be released.
func (b *BoundedCachedCompressors) AcquireGzipReader() *gzip.Reader {
var reader *gzip.Reader
select {
case reader, _ = <-b.gzipReaders:
default:
// return a new unmanaged one
reader = newGzipReader()
}
return reader
}
// ReleaseGzipReader accepts a reader (does not have to be one that was cached)
// only when the cache has room for it. It will ignore it otherwise.
func (b *BoundedCachedCompressors) ReleaseGzipReader(r *gzip.Reader) {
// forget the unmanaged ones
if len(b.gzipReaders) < b.readersCapacity {
b.gzipReaders <- r
}
}
// AcquireZlibWriter returns an resettable *zlib.Writer. Needs to be released.
func (b *BoundedCachedCompressors) AcquireZlibWriter() *zlib.Writer {
var writer *zlib.Writer
select {
case writer, _ = <-b.zlibWriters:
default:
// return a new unmanaged one
writer = newZlibWriter()
}
return writer
}
// ReleaseZlibWriter accepts a writer (does not have to be one that was cached)
// only when the cache has room for it. It will ignore it otherwise.
func (b *BoundedCachedCompressors) ReleaseZlibWriter(w *zlib.Writer) {
// forget the unmanaged ones
if len(b.zlibWriters) < b.writersCapacity {
b.zlibWriters <- w
}
}

View File

@@ -1,91 +0,0 @@
package restful
// Copyright 2015 Ernest Micklei. All rights reserved.
// Use of this source code is governed by a license
// that can be found in the LICENSE file.
import (
"bytes"
"compress/gzip"
"compress/zlib"
"sync"
)
// SyncPoolCompessors is a CompressorProvider that use the standard sync.Pool.
type SyncPoolCompessors struct {
GzipWriterPool *sync.Pool
GzipReaderPool *sync.Pool
ZlibWriterPool *sync.Pool
}
// NewSyncPoolCompessors returns a new ("empty") SyncPoolCompessors.
func NewSyncPoolCompessors() *SyncPoolCompessors {
return &SyncPoolCompessors{
GzipWriterPool: &sync.Pool{
New: func() interface{} { return newGzipWriter() },
},
GzipReaderPool: &sync.Pool{
New: func() interface{} { return newGzipReader() },
},
ZlibWriterPool: &sync.Pool{
New: func() interface{} { return newZlibWriter() },
},
}
}
func (s *SyncPoolCompessors) AcquireGzipWriter() *gzip.Writer {
return s.GzipWriterPool.Get().(*gzip.Writer)
}
func (s *SyncPoolCompessors) ReleaseGzipWriter(w *gzip.Writer) {
s.GzipWriterPool.Put(w)
}
func (s *SyncPoolCompessors) AcquireGzipReader() *gzip.Reader {
return s.GzipReaderPool.Get().(*gzip.Reader)
}
func (s *SyncPoolCompessors) ReleaseGzipReader(r *gzip.Reader) {
s.GzipReaderPool.Put(r)
}
func (s *SyncPoolCompessors) AcquireZlibWriter() *zlib.Writer {
return s.ZlibWriterPool.Get().(*zlib.Writer)
}
func (s *SyncPoolCompessors) ReleaseZlibWriter(w *zlib.Writer) {
s.ZlibWriterPool.Put(w)
}
func newGzipWriter() *gzip.Writer {
// create with an empty bytes writer; it will be replaced before using the gzipWriter
writer, err := gzip.NewWriterLevel(new(bytes.Buffer), gzip.BestSpeed)
if err != nil {
panic(err.Error())
}
return writer
}
func newGzipReader() *gzip.Reader {
// create with an empty reader (but with GZIP header); it will be replaced before using the gzipReader
// we can safely use currentCompressProvider because it is set on package initialization.
w := currentCompressorProvider.AcquireGzipWriter()
defer currentCompressorProvider.ReleaseGzipWriter(w)
b := new(bytes.Buffer)
w.Reset(b)
w.Flush()
w.Close()
reader, err := gzip.NewReader(bytes.NewReader(b.Bytes()))
if err != nil {
panic(err.Error())
}
return reader
}
func newZlibWriter() *zlib.Writer {
writer, err := zlib.NewWriterLevel(new(bytes.Buffer), gzip.BestSpeed)
if err != nil {
panic(err.Error())
}
return writer
}

View File

@@ -1,54 +0,0 @@
package restful
// Copyright 2015 Ernest Micklei. All rights reserved.
// Use of this source code is governed by a license
// that can be found in the LICENSE file.
import (
"compress/gzip"
"compress/zlib"
)
// CompressorProvider describes a component that can provider compressors for the std methods.
type CompressorProvider interface {
// Returns a *gzip.Writer which needs to be released later.
// Before using it, call Reset().
AcquireGzipWriter() *gzip.Writer
// Releases an acquired *gzip.Writer.
ReleaseGzipWriter(w *gzip.Writer)
// Returns a *gzip.Reader which needs to be released later.
AcquireGzipReader() *gzip.Reader
// Releases an acquired *gzip.Reader.
ReleaseGzipReader(w *gzip.Reader)
// Returns a *zlib.Writer which needs to be released later.
// Before using it, call Reset().
AcquireZlibWriter() *zlib.Writer
// Releases an acquired *zlib.Writer.
ReleaseZlibWriter(w *zlib.Writer)
}
// DefaultCompressorProvider is the actual provider of compressors (zlib or gzip).
var currentCompressorProvider CompressorProvider
func init() {
currentCompressorProvider = NewSyncPoolCompessors()
}
// CurrentCompressorProvider returns the current CompressorProvider.
// It is initialized using a SyncPoolCompessors.
func CurrentCompressorProvider() CompressorProvider {
return currentCompressorProvider
}
// SetCompressorProvider sets the actual provider of compressors (zlib or gzip).
func SetCompressorProvider(p CompressorProvider) {
if p == nil {
panic("cannot set compressor provider to nil")
}
currentCompressorProvider = p
}

View File

@@ -1,32 +0,0 @@
package restful
// Copyright 2013 Ernest Micklei. All rights reserved.
// Use of this source code is governed by a license
// that can be found in the LICENSE file.
const (
MIME_XML = "application/xml" // Accept or Content-Type used in Consumes() and/or Produces()
MIME_JSON = "application/json" // Accept or Content-Type used in Consumes() and/or Produces()
MIME_ZIP = "application/zip" // Accept or Content-Type used in Consumes() and/or Produces()
MIME_OCTET = "application/octet-stream" // If Content-Type is not present in request, use the default
HEADER_Allow = "Allow"
HEADER_Accept = "Accept"
HEADER_Origin = "Origin"
HEADER_ContentType = "Content-Type"
HEADER_ContentDisposition = "Content-Disposition"
HEADER_LastModified = "Last-Modified"
HEADER_AcceptEncoding = "Accept-Encoding"
HEADER_ContentEncoding = "Content-Encoding"
HEADER_AccessControlExposeHeaders = "Access-Control-Expose-Headers"
HEADER_AccessControlRequestMethod = "Access-Control-Request-Method"
HEADER_AccessControlRequestHeaders = "Access-Control-Request-Headers"
HEADER_AccessControlAllowMethods = "Access-Control-Allow-Methods"
HEADER_AccessControlAllowOrigin = "Access-Control-Allow-Origin"
HEADER_AccessControlAllowCredentials = "Access-Control-Allow-Credentials"
HEADER_AccessControlAllowHeaders = "Access-Control-Allow-Headers"
HEADER_AccessControlMaxAge = "Access-Control-Max-Age"
ENCODING_GZIP = "gzip"
ENCODING_DEFLATE = "deflate"
)

View File

@@ -1,450 +0,0 @@
package restful
// Copyright 2013 Ernest Micklei. All rights reserved.
// Use of this source code is governed by a license
// that can be found in the LICENSE file.
import (
"bytes"
"errors"
"fmt"
"net/http"
"os"
"runtime"
"strings"
"sync"
"github.com/emicklei/go-restful/v3/log"
)
// Container holds a collection of WebServices and a http.ServeMux to dispatch http requests.
// The requests are further dispatched to routes of WebServices using a RouteSelector
type Container struct {
webServicesLock sync.RWMutex
webServices []*WebService
ServeMux *http.ServeMux
isRegisteredOnRoot bool
containerFilters []FilterFunction
doNotRecover bool // default is true
recoverHandleFunc RecoverHandleFunction
serviceErrorHandleFunc ServiceErrorHandleFunction
router RouteSelector // default is a CurlyRouter (RouterJSR311 is a slower alternative)
contentEncodingEnabled bool // default is false
}
// NewContainer creates a new Container using a new ServeMux and default router (CurlyRouter)
func NewContainer() *Container {
return &Container{
webServices: []*WebService{},
ServeMux: http.NewServeMux(),
isRegisteredOnRoot: false,
containerFilters: []FilterFunction{},
doNotRecover: true,
recoverHandleFunc: logStackOnRecover,
serviceErrorHandleFunc: writeServiceError,
router: CurlyRouter{},
contentEncodingEnabled: false}
}
// RecoverHandleFunction declares functions that can be used to handle a panic situation.
// The first argument is what recover() returns. The second must be used to communicate an error response.
type RecoverHandleFunction func(interface{}, http.ResponseWriter)
// RecoverHandler changes the default function (logStackOnRecover) to be called
// when a panic is detected. DoNotRecover must be have its default value (=false).
func (c *Container) RecoverHandler(handler RecoverHandleFunction) {
c.recoverHandleFunc = handler
}
// ServiceErrorHandleFunction declares functions that can be used to handle a service error situation.
// The first argument is the service error, the second is the request that resulted in the error and
// the third must be used to communicate an error response.
type ServiceErrorHandleFunction func(ServiceError, *Request, *Response)
// ServiceErrorHandler changes the default function (writeServiceError) to be called
// when a ServiceError is detected.
func (c *Container) ServiceErrorHandler(handler ServiceErrorHandleFunction) {
c.serviceErrorHandleFunc = handler
}
// DoNotRecover controls whether panics will be caught to return HTTP 500.
// If set to true, Route functions are responsible for handling any error situation.
// Default value is true.
func (c *Container) DoNotRecover(doNot bool) {
c.doNotRecover = doNot
}
// Router changes the default Router (currently CurlyRouter)
func (c *Container) Router(aRouter RouteSelector) {
c.router = aRouter
}
// EnableContentEncoding (default=false) allows for GZIP or DEFLATE encoding of responses.
func (c *Container) EnableContentEncoding(enabled bool) {
c.contentEncodingEnabled = enabled
}
// Add a WebService to the Container. It will detect duplicate root paths and exit in that case.
func (c *Container) Add(service *WebService) *Container {
c.webServicesLock.Lock()
defer c.webServicesLock.Unlock()
// if rootPath was not set then lazy initialize it
if len(service.rootPath) == 0 {
service.Path("/")
}
// cannot have duplicate root paths
for _, each := range c.webServices {
if each.RootPath() == service.RootPath() {
log.Printf("WebService with duplicate root path detected:['%v']", each)
os.Exit(1)
}
}
// If not registered on root then add specific mapping
if !c.isRegisteredOnRoot {
c.isRegisteredOnRoot = c.addHandler(service, c.ServeMux)
}
c.webServices = append(c.webServices, service)
return c
}
// addHandler may set a new HandleFunc for the serveMux
// this function must run inside the critical region protected by the webServicesLock.
// returns true if the function was registered on root ("/")
func (c *Container) addHandler(service *WebService, serveMux *http.ServeMux) bool {
pattern := fixedPrefixPath(service.RootPath())
// check if root path registration is needed
if "/" == pattern || "" == pattern {
serveMux.HandleFunc("/", c.dispatch)
return true
}
// detect if registration already exists
alreadyMapped := false
for _, each := range c.webServices {
if each.RootPath() == service.RootPath() {
alreadyMapped = true
break
}
}
if !alreadyMapped {
serveMux.HandleFunc(pattern, c.dispatch)
if !strings.HasSuffix(pattern, "/") {
serveMux.HandleFunc(pattern+"/", c.dispatch)
}
}
return false
}
func (c *Container) Remove(ws *WebService) error {
if c.ServeMux == http.DefaultServeMux {
errMsg := fmt.Sprintf("cannot remove a WebService from a Container using the DefaultServeMux: ['%v']", ws)
log.Print(errMsg)
return errors.New(errMsg)
}
c.webServicesLock.Lock()
defer c.webServicesLock.Unlock()
// build a new ServeMux and re-register all WebServices
newServeMux := http.NewServeMux()
newServices := []*WebService{}
newIsRegisteredOnRoot := false
for _, each := range c.webServices {
if each.rootPath != ws.rootPath {
// If not registered on root then add specific mapping
if !newIsRegisteredOnRoot {
newIsRegisteredOnRoot = c.addHandler(each, newServeMux)
}
newServices = append(newServices, each)
}
}
c.webServices, c.ServeMux, c.isRegisteredOnRoot = newServices, newServeMux, newIsRegisteredOnRoot
return nil
}
// logStackOnRecover is the default RecoverHandleFunction and is called
// when DoNotRecover is false and the recoverHandleFunc is not set for the container.
// Default implementation logs the stacktrace and writes the stacktrace on the response.
// This may be a security issue as it exposes sourcecode information.
func logStackOnRecover(panicReason interface{}, httpWriter http.ResponseWriter) {
var buffer bytes.Buffer
buffer.WriteString(fmt.Sprintf("recover from panic situation: - %v\r\n", panicReason))
for i := 2; ; i += 1 {
_, file, line, ok := runtime.Caller(i)
if !ok {
break
}
buffer.WriteString(fmt.Sprintf(" %s:%d\r\n", file, line))
}
log.Print(buffer.String())
httpWriter.WriteHeader(http.StatusInternalServerError)
httpWriter.Write(buffer.Bytes())
}
// writeServiceError is the default ServiceErrorHandleFunction and is called
// when a ServiceError is returned during route selection. Default implementation
// calls resp.WriteErrorString(err.Code, err.Message)
func writeServiceError(err ServiceError, req *Request, resp *Response) {
for header, values := range err.Header {
for _, value := range values {
resp.Header().Add(header, value)
}
}
resp.WriteErrorString(err.Code, err.Message)
}
// Dispatch the incoming Http Request to a matching WebService.
func (c *Container) Dispatch(httpWriter http.ResponseWriter, httpRequest *http.Request) {
if httpWriter == nil {
panic("httpWriter cannot be nil")
}
if httpRequest == nil {
panic("httpRequest cannot be nil")
}
c.dispatch(httpWriter, httpRequest)
}
// Dispatch the incoming Http Request to a matching WebService.
func (c *Container) dispatch(httpWriter http.ResponseWriter, httpRequest *http.Request) {
// so we can assign a compressing one later
writer := httpWriter
// CompressingResponseWriter should be closed after all operations are done
defer func() {
if compressWriter, ok := writer.(*CompressingResponseWriter); ok {
compressWriter.Close()
}
}()
// Instal panic recovery unless told otherwise
if !c.doNotRecover { // catch all for 500 response
defer func() {
if r := recover(); r != nil {
c.recoverHandleFunc(r, writer)
return
}
}()
}
// Find best match Route ; err is non nil if no match was found
var webService *WebService
var route *Route
var err error
func() {
c.webServicesLock.RLock()
defer c.webServicesLock.RUnlock()
webService, route, err = c.router.SelectRoute(
c.webServices,
httpRequest)
}()
if err != nil {
// a non-200 response (may be compressed) has already been written
// run container filters anyway ; they should not touch the response...
chain := FilterChain{Filters: c.containerFilters, Target: func(req *Request, resp *Response) {
switch err.(type) {
case ServiceError:
ser := err.(ServiceError)
c.serviceErrorHandleFunc(ser, req, resp)
}
// TODO
}}
chain.ProcessFilter(NewRequest(httpRequest), NewResponse(writer))
return
}
// Unless httpWriter is already an CompressingResponseWriter see if we need to install one
if _, isCompressing := httpWriter.(*CompressingResponseWriter); !isCompressing {
// Detect if compression is needed
// assume without compression, test for override
contentEncodingEnabled := c.contentEncodingEnabled
if route != nil && route.contentEncodingEnabled != nil {
contentEncodingEnabled = *route.contentEncodingEnabled
}
if contentEncodingEnabled {
doCompress, encoding := wantsCompressedResponse(httpRequest, httpWriter)
if doCompress {
var err error
writer, err = NewCompressingResponseWriter(httpWriter, encoding)
if err != nil {
log.Print("unable to install compressor: ", err)
httpWriter.WriteHeader(http.StatusInternalServerError)
return
}
}
}
}
pathProcessor, routerProcessesPath := c.router.(PathProcessor)
if !routerProcessesPath {
pathProcessor = defaultPathProcessor{}
}
pathParams := pathProcessor.ExtractParameters(route, webService, httpRequest.URL.Path)
wrappedRequest, wrappedResponse := route.wrapRequestResponse(writer, httpRequest, pathParams)
// pass through filters (if any)
if size := len(c.containerFilters) + len(webService.filters) + len(route.Filters); size > 0 {
// compose filter chain
allFilters := make([]FilterFunction, 0, size)
allFilters = append(allFilters, c.containerFilters...)
allFilters = append(allFilters, webService.filters...)
allFilters = append(allFilters, route.Filters...)
chain := FilterChain{
Filters: allFilters,
Target: route.Function,
ParameterDocs: route.ParameterDocs,
Operation: route.Operation,
}
chain.ProcessFilter(wrappedRequest, wrappedResponse)
} else {
// no filters, handle request by route
route.Function(wrappedRequest, wrappedResponse)
}
}
// fixedPrefixPath returns the fixed part of the partspec ; it may include template vars {}
func fixedPrefixPath(pathspec string) string {
varBegin := strings.Index(pathspec, "{")
if -1 == varBegin {
return pathspec
}
return pathspec[:varBegin]
}
// ServeHTTP implements net/http.Handler therefore a Container can be a Handler in a http.Server
func (c *Container) ServeHTTP(httpWriter http.ResponseWriter, httpRequest *http.Request) {
// Skip, if content encoding is disabled
if !c.contentEncodingEnabled {
c.ServeMux.ServeHTTP(httpWriter, httpRequest)
return
}
// content encoding is enabled
// Skip, if httpWriter is already an CompressingResponseWriter
if _, ok := httpWriter.(*CompressingResponseWriter); ok {
c.ServeMux.ServeHTTP(httpWriter, httpRequest)
return
}
writer := httpWriter
// CompressingResponseWriter should be closed after all operations are done
defer func() {
if compressWriter, ok := writer.(*CompressingResponseWriter); ok {
compressWriter.Close()
}
}()
doCompress, encoding := wantsCompressedResponse(httpRequest, httpWriter)
if doCompress {
var err error
writer, err = NewCompressingResponseWriter(httpWriter, encoding)
if err != nil {
log.Print("unable to install compressor: ", err)
httpWriter.WriteHeader(http.StatusInternalServerError)
return
}
}
c.ServeMux.ServeHTTP(writer, httpRequest)
}
// Handle registers the handler for the given pattern. If a handler already exists for pattern, Handle panics.
func (c *Container) Handle(pattern string, handler http.Handler) {
c.ServeMux.Handle(pattern, http.HandlerFunc(func(httpWriter http.ResponseWriter, httpRequest *http.Request) {
// Skip, if httpWriter is already an CompressingResponseWriter
if _, ok := httpWriter.(*CompressingResponseWriter); ok {
handler.ServeHTTP(httpWriter, httpRequest)
return
}
writer := httpWriter
// CompressingResponseWriter should be closed after all operations are done
defer func() {
if compressWriter, ok := writer.(*CompressingResponseWriter); ok {
compressWriter.Close()
}
}()
if c.contentEncodingEnabled {
doCompress, encoding := wantsCompressedResponse(httpRequest, httpWriter)
if doCompress {
var err error
writer, err = NewCompressingResponseWriter(httpWriter, encoding)
if err != nil {
log.Print("unable to install compressor: ", err)
httpWriter.WriteHeader(http.StatusInternalServerError)
return
}
}
}
handler.ServeHTTP(writer, httpRequest)
}))
}
// HandleWithFilter registers the handler for the given pattern.
// Container's filter chain is applied for handler.
// If a handler already exists for pattern, HandleWithFilter panics.
func (c *Container) HandleWithFilter(pattern string, handler http.Handler) {
f := func(httpResponse http.ResponseWriter, httpRequest *http.Request) {
if len(c.containerFilters) == 0 {
handler.ServeHTTP(httpResponse, httpRequest)
return
}
chain := FilterChain{Filters: c.containerFilters, Target: func(req *Request, resp *Response) {
handler.ServeHTTP(resp, req.Request)
}}
chain.ProcessFilter(NewRequest(httpRequest), NewResponse(httpResponse))
}
c.Handle(pattern, http.HandlerFunc(f))
}
// Filter appends a container FilterFunction. These are called before dispatching
// a http.Request to a WebService from the container
func (c *Container) Filter(filter FilterFunction) {
c.containerFilters = append(c.containerFilters, filter)
}
// RegisteredWebServices returns the collections of added WebServices
func (c *Container) RegisteredWebServices() []*WebService {
c.webServicesLock.RLock()
defer c.webServicesLock.RUnlock()
result := make([]*WebService, len(c.webServices))
for ix := range c.webServices {
result[ix] = c.webServices[ix]
}
return result
}
// computeAllowedMethods returns a list of HTTP methods that are valid for a Request
func (c *Container) computeAllowedMethods(req *Request) []string {
// Go through all RegisteredWebServices() and all its Routes to collect the options
methods := []string{}
requestPath := req.Request.URL.Path
for _, ws := range c.RegisteredWebServices() {
matches := ws.pathExpr.Matcher.FindStringSubmatch(requestPath)
if matches != nil {
finalMatch := matches[len(matches)-1]
for _, rt := range ws.Routes() {
matches := rt.pathExpr.Matcher.FindStringSubmatch(finalMatch)
if matches != nil {
lastMatch := matches[len(matches)-1]
if lastMatch == "" || lastMatch == "/" { // do not include if value is neither empty nor /.
methods = append(methods, rt.Method)
}
}
}
}
}
// methods = append(methods, "OPTIONS") not sure about this
return methods
}
// newBasicRequestResponse creates a pair of Request,Response from its http versions.
// It is basic because no parameter or (produces) content-type information is given.
func newBasicRequestResponse(httpWriter http.ResponseWriter, httpRequest *http.Request) (*Request, *Response) {
resp := NewResponse(httpWriter)
resp.requestAccept = httpRequest.Header.Get(HEADER_Accept)
return NewRequest(httpRequest), resp
}

View File

@@ -1,193 +0,0 @@
package restful
// Copyright 2013 Ernest Micklei. All rights reserved.
// Use of this source code is governed by a license
// that can be found in the LICENSE file.
import (
"regexp"
"strconv"
"strings"
)
// CrossOriginResourceSharing is used to create a Container Filter that implements CORS.
// Cross-origin resource sharing (CORS) is a mechanism that allows JavaScript on a web page
// to make XMLHttpRequests to another domain, not the domain the JavaScript originated from.
//
// http://en.wikipedia.org/wiki/Cross-origin_resource_sharing
// http://enable-cors.org/server.html
// http://www.html5rocks.com/en/tutorials/cors/#toc-handling-a-not-so-simple-request
type CrossOriginResourceSharing struct {
ExposeHeaders []string // list of Header names
// AllowedHeaders is alist of Header names. Checking is case-insensitive.
// The list may contain the special wildcard string ".*" ; all is allowed
AllowedHeaders []string
// AllowedDomains is a list of allowed values for Http Origin.
// The list may contain the special wildcard string ".*" ; all is allowed
// If empty all are allowed.
AllowedDomains []string
// AllowedDomainFunc is optional and is a function that will do the check
// when the origin is not part of the AllowedDomains and it does not contain the wildcard ".*".
AllowedDomainFunc func(origin string) bool
// AllowedMethods is either empty or has a list of http methods names. Checking is case-insensitive.
AllowedMethods []string
MaxAge int // number of seconds before requiring new Options request
CookiesAllowed bool
Container *Container
allowedOriginPatterns []*regexp.Regexp // internal field for origin regexp check.
}
// Filter is a filter function that implements the CORS flow as documented on http://enable-cors.org/server.html
// and http://www.html5rocks.com/static/images/cors_server_flowchart.png
func (c CrossOriginResourceSharing) Filter(req *Request, resp *Response, chain *FilterChain) {
origin := req.Request.Header.Get(HEADER_Origin)
if len(origin) == 0 {
if trace {
traceLogger.Print("no Http header Origin set")
}
chain.ProcessFilter(req, resp)
return
}
if !c.isOriginAllowed(origin) { // check whether this origin is allowed
if trace {
traceLogger.Printf("HTTP Origin:%s is not part of %v, neither matches any part of %v", origin, c.AllowedDomains, c.allowedOriginPatterns)
}
chain.ProcessFilter(req, resp)
return
}
if req.Request.Method != "OPTIONS" {
c.doActualRequest(req, resp)
chain.ProcessFilter(req, resp)
return
}
if acrm := req.Request.Header.Get(HEADER_AccessControlRequestMethod); acrm != "" {
c.doPreflightRequest(req, resp)
} else {
c.doActualRequest(req, resp)
chain.ProcessFilter(req, resp)
return
}
}
func (c CrossOriginResourceSharing) doActualRequest(req *Request, resp *Response) {
c.setOptionsHeaders(req, resp)
// continue processing the response
}
func (c *CrossOriginResourceSharing) doPreflightRequest(req *Request, resp *Response) {
if len(c.AllowedMethods) == 0 {
if c.Container == nil {
c.AllowedMethods = DefaultContainer.computeAllowedMethods(req)
} else {
c.AllowedMethods = c.Container.computeAllowedMethods(req)
}
}
acrm := req.Request.Header.Get(HEADER_AccessControlRequestMethod)
if !c.isValidAccessControlRequestMethod(acrm, c.AllowedMethods) {
if trace {
traceLogger.Printf("Http header %s:%s is not in %v",
HEADER_AccessControlRequestMethod,
acrm,
c.AllowedMethods)
}
return
}
acrhs := req.Request.Header.Get(HEADER_AccessControlRequestHeaders)
if len(acrhs) > 0 {
for _, each := range strings.Split(acrhs, ",") {
if !c.isValidAccessControlRequestHeader(strings.Trim(each, " ")) {
if trace {
traceLogger.Printf("Http header %s:%s is not in %v",
HEADER_AccessControlRequestHeaders,
acrhs,
c.AllowedHeaders)
}
return
}
}
}
resp.AddHeader(HEADER_AccessControlAllowMethods, strings.Join(c.AllowedMethods, ","))
resp.AddHeader(HEADER_AccessControlAllowHeaders, acrhs)
c.setOptionsHeaders(req, resp)
// return http 200 response, no body
}
func (c CrossOriginResourceSharing) setOptionsHeaders(req *Request, resp *Response) {
c.checkAndSetExposeHeaders(resp)
c.setAllowOriginHeader(req, resp)
c.checkAndSetAllowCredentials(resp)
if c.MaxAge > 0 {
resp.AddHeader(HEADER_AccessControlMaxAge, strconv.Itoa(c.MaxAge))
}
}
func (c CrossOriginResourceSharing) isOriginAllowed(origin string) bool {
if len(origin) == 0 {
return false
}
lowerOrigin := strings.ToLower(origin)
if len(c.AllowedDomains) == 0 {
if c.AllowedDomainFunc != nil {
return c.AllowedDomainFunc(lowerOrigin)
}
return true
}
// exact match on each allowed domain
for _, domain := range c.AllowedDomains {
if domain == ".*" || strings.ToLower(domain) == lowerOrigin {
return true
}
}
if c.AllowedDomainFunc != nil {
return c.AllowedDomainFunc(origin)
}
return false
}
func (c CrossOriginResourceSharing) setAllowOriginHeader(req *Request, resp *Response) {
origin := req.Request.Header.Get(HEADER_Origin)
if c.isOriginAllowed(origin) {
resp.AddHeader(HEADER_AccessControlAllowOrigin, origin)
}
}
func (c CrossOriginResourceSharing) checkAndSetExposeHeaders(resp *Response) {
if len(c.ExposeHeaders) > 0 {
resp.AddHeader(HEADER_AccessControlExposeHeaders, strings.Join(c.ExposeHeaders, ","))
}
}
func (c CrossOriginResourceSharing) checkAndSetAllowCredentials(resp *Response) {
if c.CookiesAllowed {
resp.AddHeader(HEADER_AccessControlAllowCredentials, "true")
}
}
func (c CrossOriginResourceSharing) isValidAccessControlRequestMethod(method string, allowedMethods []string) bool {
for _, each := range allowedMethods {
if each == method {
return true
}
}
return false
}
func (c CrossOriginResourceSharing) isValidAccessControlRequestHeader(header string) bool {
for _, each := range c.AllowedHeaders {
if strings.ToLower(each) == strings.ToLower(header) {
return true
}
if each == "*" {
return true
}
}
return false
}

View File

@@ -1,2 +0,0 @@
go test -coverprofile=coverage.out
go tool cover -html=coverage.out

View File

@@ -1,173 +0,0 @@
package restful
// Copyright 2013 Ernest Micklei. All rights reserved.
// Use of this source code is governed by a license
// that can be found in the LICENSE file.
import (
"net/http"
"regexp"
"sort"
"strings"
)
// CurlyRouter expects Routes with paths that contain zero or more parameters in curly brackets.
type CurlyRouter struct{}
// SelectRoute is part of the Router interface and returns the best match
// for the WebService and its Route for the given Request.
func (c CurlyRouter) SelectRoute(
webServices []*WebService,
httpRequest *http.Request) (selectedService *WebService, selected *Route, err error) {
requestTokens := tokenizePath(httpRequest.URL.Path)
detectedService := c.detectWebService(requestTokens, webServices)
if detectedService == nil {
if trace {
traceLogger.Printf("no WebService was found to match URL path:%s\n", httpRequest.URL.Path)
}
return nil, nil, NewError(http.StatusNotFound, "404: Page Not Found")
}
candidateRoutes := c.selectRoutes(detectedService, requestTokens)
if len(candidateRoutes) == 0 {
if trace {
traceLogger.Printf("no Route in WebService with path %s was found to match URL path:%s\n", detectedService.rootPath, httpRequest.URL.Path)
}
return detectedService, nil, NewError(http.StatusNotFound, "404: Page Not Found")
}
selectedRoute, err := c.detectRoute(candidateRoutes, httpRequest)
if selectedRoute == nil {
return detectedService, nil, err
}
return detectedService, selectedRoute, nil
}
// selectRoutes return a collection of Route from a WebService that matches the path tokens from the request.
func (c CurlyRouter) selectRoutes(ws *WebService, requestTokens []string) sortableCurlyRoutes {
candidates := make(sortableCurlyRoutes, 0, 8)
for _, each := range ws.routes {
matches, paramCount, staticCount := c.matchesRouteByPathTokens(each.pathParts, requestTokens, each.hasCustomVerb)
if matches {
candidates.add(curlyRoute{each, paramCount, staticCount}) // TODO make sure Routes() return pointers?
}
}
sort.Sort(candidates)
return candidates
}
// matchesRouteByPathTokens computes whether it matches, howmany parameters do match and what the number of static path elements are.
func (c CurlyRouter) matchesRouteByPathTokens(routeTokens, requestTokens []string, routeHasCustomVerb bool) (matches bool, paramCount int, staticCount int) {
if len(routeTokens) < len(requestTokens) {
// proceed in matching only if last routeToken is wildcard
count := len(routeTokens)
if count == 0 || !strings.HasSuffix(routeTokens[count-1], "*}") {
return false, 0, 0
}
// proceed
}
for i, routeToken := range routeTokens {
if i == len(requestTokens) {
// reached end of request path
return false, 0, 0
}
requestToken := requestTokens[i]
if routeHasCustomVerb && hasCustomVerb(routeToken){
if !isMatchCustomVerb(routeToken, requestToken) {
return false, 0, 0
}
staticCount++
requestToken = removeCustomVerb(requestToken)
routeToken = removeCustomVerb(routeToken)
}
if strings.HasPrefix(routeToken, "{") {
paramCount++
if colon := strings.Index(routeToken, ":"); colon != -1 {
// match by regex
matchesToken, matchesRemainder := c.regularMatchesPathToken(routeToken, colon, requestToken)
if !matchesToken {
return false, 0, 0
}
if matchesRemainder {
break
}
}
} else { // no { prefix
if requestToken != routeToken {
return false, 0, 0
}
staticCount++
}
}
return true, paramCount, staticCount
}
// regularMatchesPathToken tests whether the regular expression part of routeToken matches the requestToken or all remaining tokens
// format routeToken is {someVar:someExpression}, e.g. {zipcode:[\d][\d][\d][\d][A-Z][A-Z]}
func (c CurlyRouter) regularMatchesPathToken(routeToken string, colon int, requestToken string) (matchesToken bool, matchesRemainder bool) {
regPart := routeToken[colon+1 : len(routeToken)-1]
if regPart == "*" {
if trace {
traceLogger.Printf("wildcard parameter detected in route token %s that matches %s\n", routeToken, requestToken)
}
return true, true
}
matched, err := regexp.MatchString(regPart, requestToken)
return (matched && err == nil), false
}
var jsr311Router = RouterJSR311{}
// detectRoute selectes from a list of Route the first match by inspecting both the Accept and Content-Type
// headers of the Request. See also RouterJSR311 in jsr311.go
func (c CurlyRouter) detectRoute(candidateRoutes sortableCurlyRoutes, httpRequest *http.Request) (*Route, error) {
// tracing is done inside detectRoute
return jsr311Router.detectRoute(candidateRoutes.routes(), httpRequest)
}
// detectWebService returns the best matching webService given the list of path tokens.
// see also computeWebserviceScore
func (c CurlyRouter) detectWebService(requestTokens []string, webServices []*WebService) *WebService {
var best *WebService
score := -1
for _, each := range webServices {
matches, eachScore := c.computeWebserviceScore(requestTokens, each.pathExpr.tokens)
if matches && (eachScore > score) {
best = each
score = eachScore
}
}
return best
}
// computeWebserviceScore returns whether tokens match and
// the weighted score of the longest matching consecutive tokens from the beginning.
func (c CurlyRouter) computeWebserviceScore(requestTokens []string, tokens []string) (bool, int) {
if len(tokens) > len(requestTokens) {
return false, 0
}
score := 0
for i := 0; i < len(tokens); i++ {
each := requestTokens[i]
other := tokens[i]
if len(each) == 0 && len(other) == 0 {
score++
continue
}
if len(other) > 0 && strings.HasPrefix(other, "{") {
// no empty match
if len(each) == 0 {
return false, score
}
score += 1
} else {
// not a parameter
if each != other {
return false, score
}
score += (len(tokens) - i) * 10 //fuzzy
}
}
return true, score
}

View File

@@ -1,54 +0,0 @@
package restful
// Copyright 2013 Ernest Micklei. All rights reserved.
// Use of this source code is governed by a license
// that can be found in the LICENSE file.
// curlyRoute exits for sorting Routes by the CurlyRouter based on number of parameters and number of static path elements.
type curlyRoute struct {
route Route
paramCount int
staticCount int
}
// sortableCurlyRoutes orders by most parameters and path elements first.
type sortableCurlyRoutes []curlyRoute
func (s *sortableCurlyRoutes) add(route curlyRoute) {
*s = append(*s, route)
}
func (s sortableCurlyRoutes) routes() (routes []Route) {
routes = make([]Route, 0, len(s))
for _, each := range s {
routes = append(routes, each.route) // TODO change return type
}
return routes
}
func (s sortableCurlyRoutes) Len() int {
return len(s)
}
func (s sortableCurlyRoutes) Swap(i, j int) {
s[i], s[j] = s[j], s[i]
}
func (s sortableCurlyRoutes) Less(i, j int) bool {
a := s[j]
b := s[i]
// primary key
if a.staticCount < b.staticCount {
return true
}
if a.staticCount > b.staticCount {
return false
}
// secundary key
if a.paramCount < b.paramCount {
return true
}
if a.paramCount > b.paramCount {
return false
}
return a.route.Path < b.route.Path
}

View File

@@ -1,29 +0,0 @@
package restful
import (
"fmt"
"regexp"
)
var (
customVerbReg = regexp.MustCompile(":([A-Za-z]+)$")
)
func hasCustomVerb(routeToken string) bool {
return customVerbReg.MatchString(routeToken)
}
func isMatchCustomVerb(routeToken string, pathToken string) bool {
rs := customVerbReg.FindStringSubmatch(routeToken)
if len(rs) < 2 {
return false
}
customVerb := rs[1]
specificVerbReg := regexp.MustCompile(fmt.Sprintf(":%s$", customVerb))
return specificVerbReg.MatchString(pathToken)
}
func removeCustomVerb(str string) string {
return customVerbReg.ReplaceAllString(str, "")
}

View File

@@ -1,185 +0,0 @@
/*
Package restful , a lean package for creating REST-style WebServices without magic.
WebServices and Routes
A WebService has a collection of Route objects that dispatch incoming Http Requests to a function calls.
Typically, a WebService has a root path (e.g. /users) and defines common MIME types for its routes.
WebServices must be added to a container (see below) in order to handler Http requests from a server.
A Route is defined by a HTTP method, an URL path and (optionally) the MIME types it consumes (Content-Type) and produces (Accept).
This package has the logic to find the best matching Route and if found, call its Function.
ws := new(restful.WebService)
ws.
Path("/users").
Consumes(restful.MIME_JSON, restful.MIME_XML).
Produces(restful.MIME_JSON, restful.MIME_XML)
ws.Route(ws.GET("/{user-id}").To(u.findUser)) // u is a UserResource
...
// GET http://localhost:8080/users/1
func (u UserResource) findUser(request *restful.Request, response *restful.Response) {
id := request.PathParameter("user-id")
...
}
The (*Request, *Response) arguments provide functions for reading information from the request and writing information back to the response.
See the example https://github.com/emicklei/go-restful/blob/v3/examples/user-resource/restful-user-resource.go with a full implementation.
Regular expression matching Routes
A Route parameter can be specified using the format "uri/{var[:regexp]}" or the special version "uri/{var:*}" for matching the tail of the path.
For example, /persons/{name:[A-Z][A-Z]} can be used to restrict values for the parameter "name" to only contain capital alphabetic characters.
Regular expressions must use the standard Go syntax as described in the regexp package. (https://code.google.com/p/re2/wiki/Syntax)
This feature requires the use of a CurlyRouter.
Containers
A Container holds a collection of WebServices, Filters and a http.ServeMux for multiplexing http requests.
Using the statements "restful.Add(...) and restful.Filter(...)" will register WebServices and Filters to the Default Container.
The Default container of go-restful uses the http.DefaultServeMux.
You can create your own Container and create a new http.Server for that particular container.
container := restful.NewContainer()
server := &http.Server{Addr: ":8081", Handler: container}
Filters
A filter dynamically intercepts requests and responses to transform or use the information contained in the requests or responses.
You can use filters to perform generic logging, measurement, authentication, redirect, set response headers etc.
In the restful package there are three hooks into the request,response flow where filters can be added.
Each filter must define a FilterFunction:
func (req *restful.Request, resp *restful.Response, chain *restful.FilterChain)
Use the following statement to pass the request,response pair to the next filter or RouteFunction
chain.ProcessFilter(req, resp)
Container Filters
These are processed before any registered WebService.
// install a (global) filter for the default container (processed before any webservice)
restful.Filter(globalLogging)
WebService Filters
These are processed before any Route of a WebService.
// install a webservice filter (processed before any route)
ws.Filter(webserviceLogging).Filter(measureTime)
Route Filters
These are processed before calling the function associated with the Route.
// install 2 chained route filters (processed before calling findUser)
ws.Route(ws.GET("/{user-id}").Filter(routeLogging).Filter(NewCountFilter().routeCounter).To(findUser))
See the example https://github.com/emicklei/go-restful/blob/v3/examples/filters/restful-filters.go with full implementations.
Response Encoding
Two encodings are supported: gzip and deflate. To enable this for all responses:
restful.DefaultContainer.EnableContentEncoding(true)
If a Http request includes the Accept-Encoding header then the response content will be compressed using the specified encoding.
Alternatively, you can create a Filter that performs the encoding and install it per WebService or Route.
See the example https://github.com/emicklei/go-restful/blob/v3/examples/encoding/restful-encoding-filter.go
OPTIONS support
By installing a pre-defined container filter, your Webservice(s) can respond to the OPTIONS Http request.
Filter(OPTIONSFilter())
CORS
By installing the filter of a CrossOriginResourceSharing (CORS), your WebService(s) can handle CORS requests.
cors := CrossOriginResourceSharing{ExposeHeaders: []string{"X-My-Header"}, CookiesAllowed: false, Container: DefaultContainer}
Filter(cors.Filter)
Error Handling
Unexpected things happen. If a request cannot be processed because of a failure, your service needs to tell via the response what happened and why.
For this reason HTTP status codes exist and it is important to use the correct code in every exceptional situation.
400: Bad Request
If path or query parameters are not valid (content or type) then use http.StatusBadRequest.
404: Not Found
Despite a valid URI, the resource requested may not be available
500: Internal Server Error
If the application logic could not process the request (or write the response) then use http.StatusInternalServerError.
405: Method Not Allowed
The request has a valid URL but the method (GET,PUT,POST,...) is not allowed.
406: Not Acceptable
The request does not have or has an unknown Accept Header set for this operation.
415: Unsupported Media Type
The request does not have or has an unknown Content-Type Header set for this operation.
ServiceError
In addition to setting the correct (error) Http status code, you can choose to write a ServiceError message on the response.
Performance options
This package has several options that affect the performance of your service. It is important to understand them and how you can change it.
restful.DefaultContainer.DoNotRecover(false)
DoNotRecover controls whether panics will be caught to return HTTP 500.
If set to false, the container will recover from panics.
Default value is true
restful.SetCompressorProvider(NewBoundedCachedCompressors(20, 20))
If content encoding is enabled then the default strategy for getting new gzip/zlib writers and readers is to use a sync.Pool.
Because writers are expensive structures, performance is even more improved when using a preloaded cache. You can also inject your own implementation.
Trouble shooting
This package has the means to produce detail logging of the complete Http request matching process and filter invocation.
Enabling this feature requires you to set an implementation of restful.StdLogger (e.g. log.Logger) instance such as:
restful.TraceLogger(log.New(os.Stdout, "[restful] ", log.LstdFlags|log.Lshortfile))
Logging
The restful.SetLogger() method allows you to override the logger used by the package. By default restful
uses the standard library `log` package and logs to stdout. Different logging packages are supported as
long as they conform to `StdLogger` interface defined in the `log` sub-package, writing an adapter for your
preferred package is simple.
Resources
[project]: https://github.com/emicklei/go-restful
[examples]: https://github.com/emicklei/go-restful/blob/master/examples
[design]: http://ernestmicklei.com/2012/11/11/go-restful-api-design/
[showcases]: https://github.com/emicklei/mora, https://github.com/emicklei/landskape
(c) 2012-2015, http://ernestmicklei.com. MIT License
*/
package restful

View File

@@ -1,162 +0,0 @@
package restful
// Copyright 2015 Ernest Micklei. All rights reserved.
// Use of this source code is governed by a license
// that can be found in the LICENSE file.
import (
"encoding/xml"
"strings"
"sync"
)
// EntityReaderWriter can read and write values using an encoding such as JSON,XML.
type EntityReaderWriter interface {
// Read a serialized version of the value from the request.
// The Request may have a decompressing reader. Depends on Content-Encoding.
Read(req *Request, v interface{}) error
// Write a serialized version of the value on the response.
// The Response may have a compressing writer. Depends on Accept-Encoding.
// status should be a valid Http Status code
Write(resp *Response, status int, v interface{}) error
}
// entityAccessRegistry is a singleton
var entityAccessRegistry = &entityReaderWriters{
protection: new(sync.RWMutex),
accessors: map[string]EntityReaderWriter{},
}
// entityReaderWriters associates MIME to an EntityReaderWriter
type entityReaderWriters struct {
protection *sync.RWMutex
accessors map[string]EntityReaderWriter
}
func init() {
RegisterEntityAccessor(MIME_JSON, NewEntityAccessorJSON(MIME_JSON))
RegisterEntityAccessor(MIME_XML, NewEntityAccessorXML(MIME_XML))
}
// RegisterEntityAccessor add/overrides the ReaderWriter for encoding content with this MIME type.
func RegisterEntityAccessor(mime string, erw EntityReaderWriter) {
entityAccessRegistry.protection.Lock()
defer entityAccessRegistry.protection.Unlock()
entityAccessRegistry.accessors[mime] = erw
}
// NewEntityAccessorJSON returns a new EntityReaderWriter for accessing JSON content.
// This package is already initialized with such an accessor using the MIME_JSON contentType.
func NewEntityAccessorJSON(contentType string) EntityReaderWriter {
return entityJSONAccess{ContentType: contentType}
}
// NewEntityAccessorXML returns a new EntityReaderWriter for accessing XML content.
// This package is already initialized with such an accessor using the MIME_XML contentType.
func NewEntityAccessorXML(contentType string) EntityReaderWriter {
return entityXMLAccess{ContentType: contentType}
}
// accessorAt returns the registered ReaderWriter for this MIME type.
func (r *entityReaderWriters) accessorAt(mime string) (EntityReaderWriter, bool) {
r.protection.RLock()
defer r.protection.RUnlock()
er, ok := r.accessors[mime]
if !ok {
// retry with reverse lookup
// more expensive but we are in an exceptional situation anyway
for k, v := range r.accessors {
if strings.Contains(mime, k) {
return v, true
}
}
}
return er, ok
}
// entityXMLAccess is a EntityReaderWriter for XML encoding
type entityXMLAccess struct {
// This is used for setting the Content-Type header when writing
ContentType string
}
// Read unmarshalls the value from XML
func (e entityXMLAccess) Read(req *Request, v interface{}) error {
return xml.NewDecoder(req.Request.Body).Decode(v)
}
// Write marshalls the value to JSON and set the Content-Type Header.
func (e entityXMLAccess) Write(resp *Response, status int, v interface{}) error {
return writeXML(resp, status, e.ContentType, v)
}
// writeXML marshalls the value to JSON and set the Content-Type Header.
func writeXML(resp *Response, status int, contentType string, v interface{}) error {
if v == nil {
resp.WriteHeader(status)
// do not write a nil representation
return nil
}
if resp.prettyPrint {
// pretty output must be created and written explicitly
output, err := xml.MarshalIndent(v, " ", " ")
if err != nil {
return err
}
resp.Header().Set(HEADER_ContentType, contentType)
resp.WriteHeader(status)
_, err = resp.Write([]byte(xml.Header))
if err != nil {
return err
}
_, err = resp.Write(output)
return err
}
// not-so-pretty
resp.Header().Set(HEADER_ContentType, contentType)
resp.WriteHeader(status)
return xml.NewEncoder(resp).Encode(v)
}
// entityJSONAccess is a EntityReaderWriter for JSON encoding
type entityJSONAccess struct {
// This is used for setting the Content-Type header when writing
ContentType string
}
// Read unmarshalls the value from JSON
func (e entityJSONAccess) Read(req *Request, v interface{}) error {
decoder := NewDecoder(req.Request.Body)
decoder.UseNumber()
return decoder.Decode(v)
}
// Write marshalls the value to JSON and set the Content-Type Header.
func (e entityJSONAccess) Write(resp *Response, status int, v interface{}) error {
return writeJSON(resp, status, e.ContentType, v)
}
// write marshalls the value to JSON and set the Content-Type Header.
func writeJSON(resp *Response, status int, contentType string, v interface{}) error {
if v == nil {
resp.WriteHeader(status)
// do not write a nil representation
return nil
}
if resp.prettyPrint {
// pretty output must be created and written explicitly
output, err := MarshalIndent(v, "", " ")
if err != nil {
return err
}
resp.Header().Set(HEADER_ContentType, contentType)
resp.WriteHeader(status)
_, err = resp.Write(output)
return err
}
// not-so-pretty
resp.Header().Set(HEADER_ContentType, contentType)
resp.WriteHeader(status)
return NewEncoder(resp).Encode(v)
}

View File

@@ -1,21 +0,0 @@
package restful
// Copyright 2021 Ernest Micklei. All rights reserved.
// Use of this source code is governed by a license
// that can be found in the LICENSE file.
// ExtensionProperties provides storage of vendor extensions for entities
type ExtensionProperties struct {
// Extensions vendor extensions used to describe extra functionality
// (https://swagger.io/docs/specification/2-0/swagger-extensions/)
Extensions map[string]interface{}
}
// AddExtension adds or updates a key=value pair to the extension map.
func (ep *ExtensionProperties) AddExtension(key string, value interface{}) {
if ep.Extensions == nil {
ep.Extensions = map[string]interface{}{key: value}
} else {
ep.Extensions[key] = value
}
}

View File

@@ -1,37 +0,0 @@
package restful
// Copyright 2013 Ernest Micklei. All rights reserved.
// Use of this source code is governed by a license
// that can be found in the LICENSE file.
// FilterChain is a request scoped object to process one or more filters before calling the target RouteFunction.
type FilterChain struct {
Filters []FilterFunction // ordered list of FilterFunction
Index int // index into filters that is currently in progress
Target RouteFunction // function to call after passing all filters
ParameterDocs []*Parameter // the parameter docs for the route
Operation string // the name of the operation
}
// ProcessFilter passes the request,response pair through the next of Filters.
// Each filter can decide to proceed to the next Filter or handle the Response itself.
func (f *FilterChain) ProcessFilter(request *Request, response *Response) {
if f.Index < len(f.Filters) {
f.Index++
f.Filters[f.Index-1](request, response, f)
} else {
f.Target(request, response)
}
}
// FilterFunction definitions must call ProcessFilter on the FilterChain to pass on the control and eventually call the RouteFunction
type FilterFunction func(*Request, *Response, *FilterChain)
// NoBrowserCacheFilter is a filter function to set HTTP headers that disable browser caching
// See examples/restful-no-cache-filter.go for usage
func NoBrowserCacheFilter(req *Request, resp *Response, chain *FilterChain) {
resp.Header().Set("Cache-Control", "no-cache, no-store, must-revalidate") // HTTP 1.1.
resp.Header().Set("Pragma", "no-cache") // HTTP 1.0.
resp.Header().Set("Expires", "0") // Proxies.
chain.ProcessFilter(req, resp)
}

View File

@@ -1,21 +0,0 @@
package restful
import (
"net/http"
)
// HttpMiddlewareHandler is a function that takes a http.Handler and returns a http.Handler
type HttpMiddlewareHandler func(http.Handler) http.Handler
// HttpMiddlewareHandlerToFilter converts a HttpMiddlewareHandler to a FilterFunction.
func HttpMiddlewareHandlerToFilter(middleware HttpMiddlewareHandler) FilterFunction {
return func(req *Request, resp *Response, chain *FilterChain) {
next := http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
req.Request = r
resp.ResponseWriter = rw
chain.ProcessFilter(req, resp)
})
middleware(next).ServeHTTP(resp.ResponseWriter, req.Request)
}
}

Some files were not shown because too many files have changed in this diff Show More