Compare commits

...

13 Commits

Author SHA1 Message Date
Zettat123
0c1f2edb99 Support specifying command for services (#50)
This PR is to support overwriting the default `CMD` command of `services` containers.

This is a Gitea specific feature and GitHub Actions doesn't support this syntax.

Reviewed-on: https://gitea.com/gitea/act/pulls/50
Reviewed-by: Jason Song <i@wolfogre.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>
2023-04-23 14:55:17 +08:00
Zettat123
721857e4a0 Remove empty steps when decoding Job (#49)
Follow #48
Empty steps are invalid, so remove them when decoding `Job` from YAML.

Reviewed-on: https://gitea.com/gitea/act/pulls/49
Reviewed-by: Jason Song <i@wolfogre.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>
2023-04-21 20:21:15 +08:00
Zettat123
6b1010ad07 Fix potential panic caused by nil Step (#48)
```yml
jobs:
  job1:
    steps:
      - run: echo HelloWorld
      - # empty step
```

If a job contains an empty step, `Job.Steps` will have a nil element and will cause panic when calling `Step.String()`.

See [the code of gitea](948a9ee5e8/models/actions/task.go (L300-L301))

Reviewed-on: https://gitea.com/gitea/act/pulls/48
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>
2023-04-21 14:45:38 +08:00
Zettat123
e12252a43a Support intepolation for env of services (#47)
Reviewed-on: https://gitea.com/gitea/act/pulls/47
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>
2023-04-20 16:24:31 +08:00
Zettat123
8609522aa4 Support services options (#45)
Reviewed-on: https://gitea.com/gitea/act/pulls/45
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>
2023-04-19 21:53:57 +08:00
Zettat123
6a876c4f99 Add go build tag to docker_network.go (#44)
Fix the build failure in https://gitea.com/gitea/act_runner/actions/runs/278/jobs/0

Reviewed-on: https://gitea.com/gitea/act/pulls/44
Reviewed-by: Jason Song <i@wolfogre.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>
2023-04-19 16:19:38 +08:00
sillyguodong
de529139af Support configuration variables (#43)
related to: https://gitea.com/gitea/act_runner/issues/127

This PR make `act` support the expression like `${{ vars.YOUR_CUSTOM_VARIABLES }}`.

Reviewed-on: https://gitea.com/gitea/act/pulls/43
Reviewed-by: Jason Song <i@wolfogre.com>
Co-authored-by: sillyguodong <gedong_1994@163.com>
Co-committed-by: sillyguodong <gedong_1994@163.com>
2023-04-19 15:22:56 +08:00
Zettat123
d3a56cdb69 Support services (#42)
Replace #5

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Jason Song <i@wolfogre.com>
Reviewed-on: https://gitea.com/gitea/act/pulls/42
Reviewed-by: Jason Song <i@wolfogre.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>
2023-04-19 11:23:28 +08:00
Galen Abell
9bdddf18e0 Parse secret inputs in reusable workflows (#41)
Secrets can be passed to reusable workflows, either explicitly by key or
implicitly by `inherit`:

https://docs.github.com/en/actions/using-workflows/reusing-workflows#using-inputs-and-secrets-in-a-reusable-workflow

Reviewed-on: https://gitea.com/gitea/act/pulls/41
Reviewed-by: Jason Song <i@wolfogre.com>
Co-authored-by: Galen Abell <galen@galenabell.com>
Co-committed-by: Galen Abell <galen@galenabell.com>
2023-04-17 13:41:02 +08:00
Zettat123
ac1ba34518 Fix incorrect job result status (#40)
Fix [#24039(GitHub)](https://github.com/go-gitea/gitea/issues/24039)

At present, if a job fails in the `Set up job`, the result status of the job will still be `success`. The reason is that the `pre` steps don't call `SetJobError`, so the `jobError` will be nil when `post` steps setting the job result. See 5c4a96bcb7/pkg/runner/job_executor.go (L99)

Reviewed-on: https://gitea.com/gitea/act/pulls/40
Reviewed-by: Jason Song <i@wolfogre.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>
2023-04-14 15:42:03 +08:00
Jason Song
5c4a96bcb7 Avoid using log.Fatal in pkg/* (#39)
Follow https://github.com/nektos/act/pull/1705

Reviewed-on: https://gitea.com/gitea/act/pulls/39
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Jason Song <i@wolfogre.com>
Co-committed-by: Jason Song <i@wolfogre.com>
2023-04-07 16:31:03 +08:00
Zettat123
62abf4fe11 Add token for getting reusable workflows from local private repos (#38)
Partially fixes https://gitea.com/gitea/act_runner/issues/91

If the repository is private, we need to provide the token to the caller workflows to access the called reusable workflows from the same repository.

Reviewed-on: https://gitea.com/gitea/act/pulls/38
Reviewed-by: Jason Song <i@wolfogre.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>
2023-04-06 14:16:20 +08:00
Zettat123
cfedc518ca Add With field to jobparser.Job (#37)
Partially Fixes [gitea/act_runner#91 comment](https://gitea.com/gitea/act_runner/issues/91#issuecomment-734544)

nektos/act has added `With` to support reusable workflows (see [code](68c72b9a51/pkg/model/workflow.go (L160)))

GitHub actions also support [`jobs.<job_id>.with`](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idwith)

Reviewed-on: https://gitea.com/gitea/act/pulls/37
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>
2023-04-04 10:59:53 +08:00
25 changed files with 484 additions and 41 deletions

View File

@@ -29,6 +29,8 @@ type NewContainerInput struct {
// Gitea specific // Gitea specific
AutoRemove bool AutoRemove bool
NetworkAliases []string
} }
// FileEntry is a file to copy to a container // FileEntry is a file to copy to a container
@@ -41,6 +43,7 @@ type FileEntry struct {
// Container for managing docker run containers // Container for managing docker run containers
type Container interface { type Container interface {
Create(capAdd []string, capDrop []string) common.Executor Create(capAdd []string, capDrop []string) common.Executor
ConnectToNetwork(name string) common.Executor
Copy(destPath string, files ...*FileEntry) common.Executor Copy(destPath string, files ...*FileEntry) common.Executor
CopyDir(destPath string, srcPath string, useGitIgnore bool) common.Executor CopyDir(destPath string, srcPath string, useGitIgnore bool) common.Executor
GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error) GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error)

View File

@@ -0,0 +1,40 @@
//go:build !(WITHOUT_DOCKER || !(linux || darwin || windows))
package container
import (
"context"
"github.com/docker/docker/api/types"
"github.com/nektos/act/pkg/common"
)
func NewDockerNetworkCreateExecutor(name string) common.Executor {
return func(ctx context.Context) error {
cli, err := GetDockerClient(ctx)
if err != nil {
return err
}
_, err = cli.NetworkCreate(ctx, name, types.NetworkCreate{
Driver: "bridge",
Scope: "local",
})
if err != nil {
return err
}
return nil
}
}
func NewDockerNetworkRemoveExecutor(name string) common.Executor {
return func(ctx context.Context) error {
cli, err := GetDockerClient(ctx)
if err != nil {
return err
}
return cli.NetworkRemove(ctx, name)
}
}

View File

@@ -16,6 +16,8 @@ import (
"strconv" "strconv"
"strings" "strings"
networktypes "github.com/docker/docker/api/types/network"
"github.com/go-git/go-billy/v5/helper/polyfill" "github.com/go-git/go-billy/v5/helper/polyfill"
"github.com/go-git/go-billy/v5/osfs" "github.com/go-git/go-billy/v5/osfs"
"github.com/go-git/go-git/v5/plumbing/format/gitignore" "github.com/go-git/go-git/v5/plumbing/format/gitignore"
@@ -46,6 +48,25 @@ func NewContainer(input *NewContainerInput) ExecutionsEnvironment {
return cr return cr
} }
func (cr *containerReference) ConnectToNetwork(name string) common.Executor {
return common.
NewDebugExecutor("%sdocker network connect %s %s", logPrefix, name, cr.input.Name).
Then(
common.NewPipelineExecutor(
cr.connect(),
cr.connectToNetwork(name, cr.input.NetworkAliases),
).IfNot(common.Dryrun),
)
}
func (cr *containerReference) connectToNetwork(name string, aliases []string) common.Executor {
return func(ctx context.Context) error {
return cr.cli.NetworkConnect(ctx, name, cr.input.Name, &networktypes.EndpointSettings{
Aliases: aliases,
})
}
}
// supportsContainerImagePlatform returns true if the underlying Docker server // supportsContainerImagePlatform returns true if the underlying Docker server
// API version is 1.41 and beyond // API version is 1.41 and beyond
func supportsContainerImagePlatform(ctx context.Context, cli client.APIClient) bool { func supportsContainerImagePlatform(ctx context.Context, cli client.APIClient) bool {

View File

@@ -55,3 +55,15 @@ func NewDockerVolumeRemoveExecutor(volume string, force bool) common.Executor {
return nil return nil
} }
} }
func NewDockerNetworkCreateExecutor(name string) common.Executor {
return func(ctx context.Context) error {
return nil
}
}
func NewDockerNetworkRemoveExecutor(name string) common.Executor {
return func(ctx context.Context) error {
return nil
}
}

View File

@@ -40,6 +40,12 @@ func (e *HostEnvironment) Create(capAdd []string, capDrop []string) common.Execu
} }
} }
func (e *HostEnvironment) ConnectToNetwork(name string) common.Executor {
return func(ctx context.Context) error {
return nil
}
}
func (e *HostEnvironment) Close() common.Executor { func (e *HostEnvironment) Close() common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
return nil return nil

View File

@@ -23,6 +23,8 @@ type EvaluationEnvironment struct {
Matrix map[string]interface{} Matrix map[string]interface{}
Needs map[string]Needs Needs map[string]Needs
Inputs map[string]interface{} Inputs map[string]interface{}
Vars map[string]string
} }
type Needs struct { type Needs struct {
@@ -181,6 +183,8 @@ func (impl *interperterImpl) evaluateVariable(variableNode *actionlint.VariableN
return math.Inf(1), nil return math.Inf(1), nil
case "nan": case "nan":
return math.NaN(), nil return math.NaN(), nil
case "vars":
return impl.env.Vars, nil
default: default:
return nil, fmt.Errorf("Unavailable context: %s", variableNode.Name) return nil, fmt.Errorf("Unavailable context: %s", variableNode.Name)
} }

View File

@@ -62,6 +62,8 @@ func NewInterpeter(
Matrix: matrix, Matrix: matrix,
Needs: using, Needs: using,
Inputs: nil, // not supported yet Inputs: nil, // not supported yet
Vars: nil,
} }
config := exprparser.Config{ config := exprparser.Config{

View File

@@ -42,7 +42,11 @@ func Parse(content []byte, options ...ParseOption) ([]*SingleWorkflow, error) {
} }
for i, id := range ids { for i, id := range ids {
job := jobs[i] job := jobs[i]
for _, matrix := range getMatrixes(origin.GetJob(id)) { matricxes, err := getMatrixes(origin.GetJob(id))
if err != nil {
return nil, fmt.Errorf("getMatrixes: %w", err)
}
for _, matrix := range matricxes {
job := job.Clone() job := job.Clone()
if job.Name == "" { if job.Name == "" {
job.Name = id job.Name = id
@@ -89,12 +93,15 @@ type parseContext struct {
type ParseOption func(c *parseContext) type ParseOption func(c *parseContext)
func getMatrixes(job *model.Job) []map[string]interface{} { func getMatrixes(job *model.Job) ([]map[string]interface{}, error) {
ret := job.GetMatrixes() ret, err := job.GetMatrixes()
if err != nil {
return nil, fmt.Errorf("GetMatrixes: %w", err)
}
sort.Slice(ret, func(i, j int) bool { sort.Slice(ret, func(i, j int) bool {
return matrixName(ret[i]) < matrixName(ret[j]) return matrixName(ret[i]) < matrixName(ret[j])
}) })
return ret return ret, nil
} }
func encodeMatrix(matrix map[string]interface{}) yaml.Node { func encodeMatrix(matrix map[string]interface{}) yaml.Node {

View File

@@ -32,6 +32,21 @@ func TestParse(t *testing.T) {
options: nil, options: nil,
wantErr: false, wantErr: false,
}, },
{
name: "has_with",
options: nil,
wantErr: false,
},
{
name: "has_secrets",
options: nil,
wantErr: false,
},
{
name: "empty_step",
options: nil,
wantErr: false,
},
} }
for _, tt := range tests { for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {

View File

@@ -40,6 +40,13 @@ func (w *SingleWorkflow) jobs() ([]string, []*Job, error) {
if err := item.Decode(job); err != nil { if err := item.Decode(job); err != nil {
return nil, nil, fmt.Errorf("yaml.Unmarshal: %w", err) return nil, nil, fmt.Errorf("yaml.Unmarshal: %w", err)
} }
steps := make([]*Step, 0, len(job.Steps))
for _, s := range job.Steps {
if s != nil {
steps = append(steps, s)
}
}
job.Steps = steps
jobs = append(jobs, job) jobs = append(jobs, job)
expectKey = true expectKey = true
} }
@@ -87,6 +94,8 @@ type Job struct {
Defaults Defaults `yaml:"defaults,omitempty"` Defaults Defaults `yaml:"defaults,omitempty"`
Outputs map[string]string `yaml:"outputs,omitempty"` Outputs map[string]string `yaml:"outputs,omitempty"`
Uses string `yaml:"uses,omitempty"` Uses string `yaml:"uses,omitempty"`
With map[string]interface{} `yaml:"with,omitempty"`
RawSecrets yaml.Node `yaml:"secrets,omitempty"`
} }
func (j *Job) Clone() *Job { func (j *Job) Clone() *Job {
@@ -107,6 +116,8 @@ func (j *Job) Clone() *Job {
Defaults: j.Defaults, Defaults: j.Defaults,
Outputs: j.Outputs, Outputs: j.Outputs,
Uses: j.Uses, Uses: j.Uses,
With: j.With,
RawSecrets: j.RawSecrets,
} }
} }
@@ -139,6 +150,9 @@ type Step struct {
// String gets the name of step // String gets the name of step
func (s *Step) String() string { func (s *Step) String() string {
if s == nil {
return ""
}
return (&model.Step{ return (&model.Step{
ID: s.ID, ID: s.ID,
Name: s.Name, Name: s.Name,
@@ -154,6 +168,7 @@ type ContainerSpec struct {
Volumes []string `yaml:"volumes,omitempty"` Volumes []string `yaml:"volumes,omitempty"`
Options string `yaml:"options,omitempty"` Options string `yaml:"options,omitempty"`
Credentials map[string]string `yaml:"credentials,omitempty"` Credentials map[string]string `yaml:"credentials,omitempty"`
Cmd []string `yaml:"cmd,omitempty"`
} }
type Strategy struct { type Strategy struct {

View File

@@ -0,0 +1,8 @@
name: test
jobs:
job1:
name: job1
runs-on: linux
steps:
- run: echo job-1
-

View File

@@ -0,0 +1,7 @@
name: test
jobs:
job1:
name: job1
runs-on: linux
steps:
- run: echo job-1

View File

@@ -0,0 +1,14 @@
name: test
jobs:
job1:
name: job1
runs-on: linux
uses: .gitea/workflows/build.yml
secrets:
secret: hideme
job2:
name: job2
runs-on: linux
uses: .gitea/workflows/build.yml
secrets: inherit

View File

@@ -0,0 +1,16 @@
name: test
jobs:
job1:
name: job1
runs-on: linux
uses: .gitea/workflows/build.yml
secrets:
secret: hideme
---
name: test
jobs:
job2:
name: job2
runs-on: linux
uses: .gitea/workflows/build.yml
secrets: inherit

15
pkg/jobparser/testdata/has_with.in.yaml vendored Normal file
View File

@@ -0,0 +1,15 @@
name: test
jobs:
job1:
name: job1
runs-on: linux
uses: .gitea/workflows/build.yml
with:
package: service
job2:
name: job2
runs-on: linux
uses: .gitea/workflows/build.yml
with:
package: module

View File

@@ -0,0 +1,17 @@
name: test
jobs:
job1:
name: job1
runs-on: linux
uses: .gitea/workflows/build.yml
with:
package: service
---
name: test
jobs:
job2:
name: job2
runs-on: linux
uses: .gitea/workflows/build.yml
with:
package: module

View File

@@ -271,15 +271,13 @@ func (j *Job) Container() *ContainerSpec {
switch j.RawContainer.Kind { switch j.RawContainer.Kind {
case yaml.ScalarNode: case yaml.ScalarNode:
val = new(ContainerSpec) val = new(ContainerSpec)
err := j.RawContainer.Decode(&val.Image) if !decodeNode(j.RawContainer, &val.Image) {
if err != nil { return nil
log.Fatal(err)
} }
case yaml.MappingNode: case yaml.MappingNode:
val = new(ContainerSpec) val = new(ContainerSpec)
err := j.RawContainer.Decode(val) if !decodeNode(j.RawContainer, val) {
if err != nil { return nil
log.Fatal(err)
} }
} }
return val return val
@@ -290,16 +288,14 @@ func (j *Job) Needs() []string {
switch j.RawNeeds.Kind { switch j.RawNeeds.Kind {
case yaml.ScalarNode: case yaml.ScalarNode:
var val string var val string
err := j.RawNeeds.Decode(&val) if !decodeNode(j.RawNeeds, &val) {
if err != nil { return nil
log.Fatal(err)
} }
return []string{val} return []string{val}
case yaml.SequenceNode: case yaml.SequenceNode:
var val []string var val []string
err := j.RawNeeds.Decode(&val) if !decodeNode(j.RawNeeds, &val) {
if err != nil { return nil
log.Fatal(err)
} }
return val return val
} }
@@ -311,16 +307,14 @@ func (j *Job) RunsOn() []string {
switch j.RawRunsOn.Kind { switch j.RawRunsOn.Kind {
case yaml.ScalarNode: case yaml.ScalarNode:
var val string var val string
err := j.RawRunsOn.Decode(&val) if !decodeNode(j.RawRunsOn, &val) {
if err != nil { return nil
log.Fatal(err)
} }
return []string{val} return []string{val}
case yaml.SequenceNode: case yaml.SequenceNode:
var val []string var val []string
err := j.RawRunsOn.Decode(&val) if !decodeNode(j.RawRunsOn, &val) {
if err != nil { return nil
log.Fatal(err)
} }
return val return val
} }
@@ -330,8 +324,8 @@ func (j *Job) RunsOn() []string {
func environment(yml yaml.Node) map[string]string { func environment(yml yaml.Node) map[string]string {
env := make(map[string]string) env := make(map[string]string)
if yml.Kind == yaml.MappingNode { if yml.Kind == yaml.MappingNode {
if err := yml.Decode(&env); err != nil { if !decodeNode(yml, &env) {
log.Fatal(err) return nil
} }
} }
return env return env
@@ -346,8 +340,8 @@ func (j *Job) Environment() map[string]string {
func (j *Job) Matrix() map[string][]interface{} { func (j *Job) Matrix() map[string][]interface{} {
if j.Strategy.RawMatrix.Kind == yaml.MappingNode { if j.Strategy.RawMatrix.Kind == yaml.MappingNode {
var val map[string][]interface{} var val map[string][]interface{}
if err := j.Strategy.RawMatrix.Decode(&val); err != nil { if !decodeNode(j.Strategy.RawMatrix, &val) {
log.Fatal(err) return nil
} }
return val return val
} }
@@ -358,7 +352,7 @@ func (j *Job) Matrix() map[string][]interface{} {
// It skips includes and hard fails excludes for non-existing keys // It skips includes and hard fails excludes for non-existing keys
// //
//nolint:gocyclo //nolint:gocyclo
func (j *Job) GetMatrixes() []map[string]interface{} { func (j *Job) GetMatrixes() ([]map[string]interface{}, error) {
matrixes := make([]map[string]interface{}, 0) matrixes := make([]map[string]interface{}, 0)
if j.Strategy != nil { if j.Strategy != nil {
j.Strategy.FailFast = j.Strategy.GetFailFast() j.Strategy.FailFast = j.Strategy.GetFailFast()
@@ -409,7 +403,7 @@ func (j *Job) GetMatrixes() []map[string]interface{} {
excludes = append(excludes, e) excludes = append(excludes, e)
} else { } else {
// We fail completely here because that's what GitHub does for non-existing matrix keys, fail on exclude, silent skip on include // We fail completely here because that's what GitHub does for non-existing matrix keys, fail on exclude, silent skip on include
log.Fatalf("The workflow is not valid. Matrix exclude key '%s' does not match any key within the matrix", k) return nil, fmt.Errorf("the workflow is not valid. Matrix exclude key %q does not match any key within the matrix", k)
} }
} }
} }
@@ -454,7 +448,7 @@ func (j *Job) GetMatrixes() []map[string]interface{} {
} else { } else {
matrixes = append(matrixes, make(map[string]interface{})) matrixes = append(matrixes, make(map[string]interface{}))
} }
return matrixes return matrixes, nil
} }
func commonKeysMatch(a map[string]interface{}, b map[string]interface{}) bool { func commonKeysMatch(a map[string]interface{}, b map[string]interface{}) bool {
@@ -528,6 +522,9 @@ type ContainerSpec struct {
Args string Args string
Name string Name string
Reuse bool Reuse bool
// Gitea specific
Cmd []string `yaml:"cmd"`
} }
// Step is the structure of one step in a job // Step is the structure of one step in a job
@@ -699,3 +696,17 @@ func (w *Workflow) GetJobIDs() []string {
} }
return ids return ids
} }
var OnDecodeNodeError = func(node yaml.Node, out interface{}, err error) {
log.Fatalf("Failed to decode node %v into %T: %v", node, out, err)
}
func decodeNode(node yaml.Node, out interface{}) bool {
if err := node.Decode(out); err != nil {
if OnDecodeNodeError != nil {
OnDecodeNodeError(node, out, err)
}
return false
}
return true
}

View File

@@ -332,25 +332,33 @@ func TestReadWorkflow_Strategy(t *testing.T) {
wf := p.Stages[0].Runs[0].Workflow wf := p.Stages[0].Runs[0].Workflow
job := wf.Jobs["strategy-only-max-parallel"] job := wf.Jobs["strategy-only-max-parallel"]
assert.Equal(t, job.GetMatrixes(), []map[string]interface{}{{}}) matrixes, err := job.GetMatrixes()
assert.NoError(t, err)
assert.Equal(t, matrixes, []map[string]interface{}{{}})
assert.Equal(t, job.Matrix(), map[string][]interface{}(nil)) assert.Equal(t, job.Matrix(), map[string][]interface{}(nil))
assert.Equal(t, job.Strategy.MaxParallel, 2) assert.Equal(t, job.Strategy.MaxParallel, 2)
assert.Equal(t, job.Strategy.FailFast, true) assert.Equal(t, job.Strategy.FailFast, true)
job = wf.Jobs["strategy-only-fail-fast"] job = wf.Jobs["strategy-only-fail-fast"]
assert.Equal(t, job.GetMatrixes(), []map[string]interface{}{{}}) matrixes, err = job.GetMatrixes()
assert.NoError(t, err)
assert.Equal(t, matrixes, []map[string]interface{}{{}})
assert.Equal(t, job.Matrix(), map[string][]interface{}(nil)) assert.Equal(t, job.Matrix(), map[string][]interface{}(nil))
assert.Equal(t, job.Strategy.MaxParallel, 4) assert.Equal(t, job.Strategy.MaxParallel, 4)
assert.Equal(t, job.Strategy.FailFast, false) assert.Equal(t, job.Strategy.FailFast, false)
job = wf.Jobs["strategy-no-matrix"] job = wf.Jobs["strategy-no-matrix"]
assert.Equal(t, job.GetMatrixes(), []map[string]interface{}{{}}) matrixes, err = job.GetMatrixes()
assert.NoError(t, err)
assert.Equal(t, matrixes, []map[string]interface{}{{}})
assert.Equal(t, job.Matrix(), map[string][]interface{}(nil)) assert.Equal(t, job.Matrix(), map[string][]interface{}(nil))
assert.Equal(t, job.Strategy.MaxParallel, 2) assert.Equal(t, job.Strategy.MaxParallel, 2)
assert.Equal(t, job.Strategy.FailFast, false) assert.Equal(t, job.Strategy.FailFast, false)
job = wf.Jobs["strategy-all"] job = wf.Jobs["strategy-all"]
assert.Equal(t, job.GetMatrixes(), matrixes, err = job.GetMatrixes()
assert.NoError(t, err)
assert.Equal(t, matrixes,
[]map[string]interface{}{ []map[string]interface{}{
{"datacenter": "site-c", "node-version": "14.x", "site": "staging"}, {"datacenter": "site-c", "node-version": "14.x", "site": "staging"},
{"datacenter": "site-c", "node-version": "16.x", "site": "staging"}, {"datacenter": "site-c", "node-version": "16.x", "site": "staging"},

View File

@@ -81,6 +81,8 @@ func (rc *RunContext) NewExpressionEvaluatorWithEnv(ctx context.Context, env map
Matrix: rc.Matrix, Matrix: rc.Matrix,
Needs: using, Needs: using,
Inputs: inputs, Inputs: inputs,
Vars: rc.getVarsContext(),
} }
if rc.JobContainer != nil { if rc.JobContainer != nil {
ee.Runner = rc.JobContainer.GetRunnerContext(ctx) ee.Runner = rc.JobContainer.GetRunnerContext(ctx)
@@ -130,6 +132,8 @@ func (rc *RunContext) NewStepExpressionEvaluator(ctx context.Context, step step)
// todo: should be unavailable // todo: should be unavailable
// but required to interpolate/evaluate the inputs in actions/composite // but required to interpolate/evaluate the inputs in actions/composite
Inputs: inputs, Inputs: inputs,
Vars: rc.getVarsContext(),
} }
if rc.JobContainer != nil { if rc.JobContainer != nil {
ee.Runner = rc.JobContainer.GetRunnerContext(ctx) ee.Runner = rc.JobContainer.GetRunnerContext(ctx)

View File

@@ -70,7 +70,19 @@ func newJobExecutor(info jobInfo, sf stepFactory, rc *RunContext) common.Executo
return common.NewErrorExecutor(err) return common.NewErrorExecutor(err)
} }
preSteps = append(preSteps, useStepLogger(rc, stepModel, stepStagePre, step.pre())) preExec := step.pre()
preSteps = append(preSteps, useStepLogger(rc, stepModel, stepStagePre, func(ctx context.Context) error {
logger := common.Logger(ctx)
preErr := preExec(ctx)
if preErr != nil {
logger.Errorf("%v", preErr)
common.SetJobError(ctx, preErr)
} else if ctx.Err() != nil {
logger.Errorf("%v", ctx.Err())
common.SetJobError(ctx, ctx.Err())
}
return preErr
}))
stepExec := step.main() stepExec := step.main()
steps = append(steps, useStepLogger(rc, stepModel, stepStageMain, func(ctx context.Context) error { steps = append(steps, useStepLogger(rc, stepModel, stepStageMain, func(ctx context.Context) error {
@@ -102,7 +114,21 @@ func newJobExecutor(info jobInfo, sf stepFactory, rc *RunContext) common.Executo
// always allow 1 min for stopping and removing the runner, even if we were cancelled // always allow 1 min for stopping and removing the runner, even if we were cancelled
ctx, cancel := context.WithTimeout(common.WithLogger(context.Background(), common.Logger(ctx)), time.Minute) ctx, cancel := context.WithTimeout(common.WithLogger(context.Background(), common.Logger(ctx)), time.Minute)
defer cancel() defer cancel()
logger := common.Logger(ctx)
logger.Infof("Cleaning up services for job %s", rc.JobName)
if err := rc.stopServiceContainers()(ctx); err != nil {
logger.Errorf("Error while cleaning services: %v", err)
}
logger.Infof("Cleaning up container for job %s", rc.JobName)
err = info.stopContainer()(ctx) err = info.stopContainer()(ctx)
logger.Infof("Cleaning up network for job %s", rc.JobName)
networkName := fmt.Sprintf("%s-network", rc.jobContainerName())
if err := rc.removeNetwork(networkName)(ctx); err != nil {
logger.Errorf("Error while cleaning network: %v", err)
}
} }
setJobResult(ctx, info, rc, jobError == nil) setJobResult(ctx, info, rc, jobError == nil)
setJobOutputs(ctx, rc) setJobOutputs(ctx, rc)

View File

@@ -30,8 +30,11 @@ func newLocalReusableWorkflowExecutor(rc *RunContext) common.Executor {
workflowDir := fmt.Sprintf("%s/%s", rc.ActionCacheDir(), safeFilename(uses)) workflowDir := fmt.Sprintf("%s/%s", rc.ActionCacheDir(), safeFilename(uses))
// If the repository is private, we need a token to clone it
token := rc.Config.GetToken()
return common.NewPipelineExecutor( return common.NewPipelineExecutor(
newMutexExecutor(cloneIfRequired(rc, *remoteReusableWorkflow, workflowDir)), newMutexExecutor(cloneIfRequired(rc, *remoteReusableWorkflow, workflowDir, token)),
newReusableWorkflowExecutor(rc, workflowDir, remoteReusableWorkflow.FilePath()), newReusableWorkflowExecutor(rc, workflowDir, remoteReusableWorkflow.FilePath()),
) )
} }
@@ -47,8 +50,11 @@ func newRemoteReusableWorkflowExecutor(rc *RunContext) common.Executor {
workflowDir := fmt.Sprintf("%s/%s", rc.ActionCacheDir(), safeFilename(uses)) workflowDir := fmt.Sprintf("%s/%s", rc.ActionCacheDir(), safeFilename(uses))
// FIXME: if the reusable workflow is from a private repository, we need to provide a token to access the repository.
token := ""
return common.NewPipelineExecutor( return common.NewPipelineExecutor(
newMutexExecutor(cloneIfRequired(rc, *remoteReusableWorkflow, workflowDir)), newMutexExecutor(cloneIfRequired(rc, *remoteReusableWorkflow, workflowDir, token)),
newReusableWorkflowExecutor(rc, workflowDir, remoteReusableWorkflow.FilePath()), newReusableWorkflowExecutor(rc, workflowDir, remoteReusableWorkflow.FilePath()),
) )
} }
@@ -66,7 +72,7 @@ func newMutexExecutor(executor common.Executor) common.Executor {
} }
} }
func cloneIfRequired(rc *RunContext, remoteReusableWorkflow remoteReusableWorkflow, targetDirectory string) common.Executor { func cloneIfRequired(rc *RunContext, remoteReusableWorkflow remoteReusableWorkflow, targetDirectory, token string) common.Executor {
return common.NewConditionalExecutor( return common.NewConditionalExecutor(
func(ctx context.Context) bool { func(ctx context.Context) bool {
_, err := os.Stat(targetDirectory) _, err := os.Stat(targetDirectory)
@@ -77,7 +83,7 @@ func cloneIfRequired(rc *RunContext, remoteReusableWorkflow remoteReusableWorkfl
URL: remoteReusableWorkflow.CloneURL(), URL: remoteReusableWorkflow.CloneURL(),
Ref: remoteReusableWorkflow.Ref, Ref: remoteReusableWorkflow.Ref,
Dir: targetDirectory, Dir: targetDirectory,
Token: rc.Config.Token, Token: token,
}), }),
nil, nil,
) )

View File

@@ -18,6 +18,7 @@ import (
"strings" "strings"
"time" "time"
"github.com/docker/docker/errdefs"
"github.com/mitchellh/go-homedir" "github.com/mitchellh/go-homedir"
"github.com/opencontainers/selinux/go-selinux" "github.com/opencontainers/selinux/go-selinux"
log "github.com/sirupsen/logrus" log "github.com/sirupsen/logrus"
@@ -43,6 +44,7 @@ type RunContext struct {
IntraActionState map[string]map[string]string IntraActionState map[string]map[string]string
ExprEval ExpressionEvaluator ExprEval ExpressionEvaluator
JobContainer container.ExecutionsEnvironment JobContainer container.ExecutionsEnvironment
ServiceContainers []container.ExecutionsEnvironment
OutputMappings map[MappableOutput]MappableOutput OutputMappings map[MappableOutput]MappableOutput
JobName string JobName string
ActionPath string ActionPath string
@@ -242,6 +244,50 @@ func (rc *RunContext) startJobContainer() common.Executor {
ext := container.LinuxContainerEnvironmentExtensions{} ext := container.LinuxContainerEnvironmentExtensions{}
binds, mounts := rc.GetBindsAndMounts() binds, mounts := rc.GetBindsAndMounts()
// add service containers
for name, spec := range rc.Run.Job().Services {
// interpolate env
interpolatedEnvs := make(map[string]string, len(spec.Env))
for k, v := range spec.Env {
interpolatedEnvs[k] = rc.ExprEval.Interpolate(ctx, v)
}
envs := make([]string, 0, len(interpolatedEnvs))
for k, v := range interpolatedEnvs {
envs = append(envs, fmt.Sprintf("%s=%s", k, v))
}
// interpolate cmd
interpolatedCmd := make([]string, 0, len(spec.Cmd))
for _, v := range spec.Cmd {
interpolatedCmd = append(interpolatedCmd, rc.ExprEval.Interpolate(ctx, v))
}
serviceContainerName := createSimpleContainerName(rc.jobContainerName(), name)
c := container.NewContainer(&container.NewContainerInput{
Name: serviceContainerName,
WorkingDir: ext.ToContainerPath(rc.Config.Workdir),
Image: spec.Image,
Username: username,
Password: password,
Cmd: interpolatedCmd,
Env: envs,
Mounts: map[string]string{
// TODO merge volumes
name: ext.ToContainerPath(rc.Config.Workdir),
"act-toolcache": "/toolcache",
"act-actions": "/actions",
},
Binds: binds,
Stdout: logWriter,
Stderr: logWriter,
Privileged: rc.Config.Privileged,
UsernsMode: rc.Config.UsernsMode,
Platform: rc.Config.ContainerArchitecture,
AutoRemove: rc.Config.AutoRemove,
Options: spec.Options,
NetworkAliases: []string{name},
})
rc.ServiceContainers = append(rc.ServiceContainers, c)
}
rc.cleanUpJobContainer = func(ctx context.Context) error { rc.cleanUpJobContainer = func(ctx context.Context) error {
if rc.JobContainer != nil && !rc.Config.ReuseContainers { if rc.JobContainer != nil && !rc.Config.ReuseContainers {
return rc.JobContainer.Remove(). return rc.JobContainer.Remove().
@@ -275,11 +321,24 @@ func (rc *RunContext) startJobContainer() common.Executor {
return errors.New("Failed to create job container") return errors.New("Failed to create job container")
} }
networkName := fmt.Sprintf("%s-network", rc.jobContainerName())
return common.NewPipelineExecutor( return common.NewPipelineExecutor(
rc.pullServicesImages(rc.Config.ForcePull),
rc.JobContainer.Pull(rc.Config.ForcePull), rc.JobContainer.Pull(rc.Config.ForcePull),
rc.stopServiceContainers(),
rc.stopJobContainer(), rc.stopJobContainer(),
func(ctx context.Context) error {
err := rc.removeNetwork(networkName)(ctx)
if errdefs.IsNotFound(err) {
return nil
}
return err
},
rc.createNetwork(networkName),
rc.startServiceContainers(networkName),
rc.JobContainer.Create(rc.Config.ContainerCapAdd, rc.Config.ContainerCapDrop), rc.JobContainer.Create(rc.Config.ContainerCapAdd, rc.Config.ContainerCapDrop),
rc.JobContainer.Start(false), rc.JobContainer.Start(false),
rc.JobContainer.ConnectToNetwork(networkName),
rc.JobContainer.Copy(rc.JobContainer.GetActPath()+"/", &container.FileEntry{ rc.JobContainer.Copy(rc.JobContainer.GetActPath()+"/", &container.FileEntry{
Name: "workflow/event.json", Name: "workflow/event.json",
Mode: 0o644, Mode: 0o644,
@@ -293,6 +352,18 @@ func (rc *RunContext) startJobContainer() common.Executor {
} }
} }
func (rc *RunContext) createNetwork(name string) common.Executor {
return func(ctx context.Context) error {
return container.NewDockerNetworkCreateExecutor(name)(ctx)
}
}
func (rc *RunContext) removeNetwork(name string) common.Executor {
return func(ctx context.Context) error {
return container.NewDockerNetworkRemoveExecutor(name)(ctx)
}
}
func (rc *RunContext) execJobContainer(cmd []string, env map[string]string, user, workdir string) common.Executor { func (rc *RunContext) execJobContainer(cmd []string, env map[string]string, user, workdir string) common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
return rc.JobContainer.Exec(cmd, env, user, workdir)(ctx) return rc.JobContainer.Exec(cmd, env, user, workdir)(ctx)
@@ -354,6 +425,41 @@ func (rc *RunContext) stopJobContainer() common.Executor {
} }
} }
func (rc *RunContext) pullServicesImages(forcePull bool) common.Executor {
return func(ctx context.Context) error {
execs := []common.Executor{}
for _, c := range rc.ServiceContainers {
execs = append(execs, c.Pull(forcePull))
}
return common.NewParallelExecutor(len(execs), execs...)(ctx)
}
}
func (rc *RunContext) startServiceContainers(networkName string) common.Executor {
return func(ctx context.Context) error {
execs := []common.Executor{}
for _, c := range rc.ServiceContainers {
execs = append(execs, common.NewPipelineExecutor(
c.Pull(false),
c.Create(rc.Config.ContainerCapAdd, rc.Config.ContainerCapDrop),
c.Start(false),
c.ConnectToNetwork(networkName),
))
}
return common.NewParallelExecutor(len(execs), execs...)(ctx)
}
}
func (rc *RunContext) stopServiceContainers() common.Executor {
return func(ctx context.Context) error {
execs := []common.Executor{}
for _, c := range rc.ServiceContainers {
execs = append(execs, c.Remove())
}
return common.NewParallelExecutor(len(execs), execs...)(ctx)
}
}
// Prepare the mounts and binds for the worker // Prepare the mounts and binds for the worker
// ActionCacheDir is for rc // ActionCacheDir is for rc
@@ -589,6 +695,10 @@ func (rc *RunContext) getStepsContext() map[string]*model.StepResult {
return rc.StepResults return rc.StepResults
} }
func (rc *RunContext) getVarsContext() map[string]string {
return rc.Config.Vars
}
func (rc *RunContext) getGithubContext(ctx context.Context) *model.GithubContext { func (rc *RunContext) getGithubContext(ctx context.Context) *model.GithubContext {
logger := common.Logger(ctx) logger := common.Logger(ctx)
ghc := &model.GithubContext{ ghc := &model.GithubContext{

View File

@@ -7,11 +7,11 @@ import (
"os" "os"
"time" "time"
log "github.com/sirupsen/logrus"
"github.com/nektos/act/pkg/common" "github.com/nektos/act/pkg/common"
"github.com/nektos/act/pkg/container" "github.com/nektos/act/pkg/container"
"github.com/nektos/act/pkg/model" "github.com/nektos/act/pkg/model"
log "github.com/sirupsen/logrus"
) )
// Runner provides capabilities to run GitHub actions // Runner provides capabilities to run GitHub actions
@@ -64,6 +64,15 @@ type Config struct {
DefaultActionInstance string // the default actions web site DefaultActionInstance string // the default actions web site
PlatformPicker func(labels []string) string // platform picker, it will take precedence over Platforms if isn't nil PlatformPicker func(labels []string) string // platform picker, it will take precedence over Platforms if isn't nil
JobLoggerLevel *log.Level // the level of job logger JobLoggerLevel *log.Level // the level of job logger
Vars map[string]string // the list of variables set at the repository, environment, or organization levels.
}
func (c Config) GetToken() string {
token := c.Secrets["GITHUB_TOKEN"]
if c.Secrets["GITEA_TOKEN"] != "" {
token = c.Secrets["GITEA_TOKEN"]
}
return token
} }
type caller struct { type caller struct {
@@ -128,7 +137,11 @@ func (runner *runnerImpl) NewPlanExecutor(plan *model.Plan) common.Executor {
log.Errorf("Error while evaluating matrix: %v", err) log.Errorf("Error while evaluating matrix: %v", err)
} }
} }
matrixes := job.GetMatrixes() matrixes, err := job.GetMatrixes()
if err != nil {
log.Errorf("Error while get job's matrix: %v", err)
// fall back to empty matrixes
}
maxParallel := 4 maxParallel := 4
if job.Strategy != nil { if job.Strategy != nil {
maxParallel = job.Strategy.MaxParallel maxParallel = job.Strategy.MaxParallel

View File

@@ -546,6 +546,43 @@ func TestRunEventSecrets(t *testing.T) {
tjfi.runTest(context.Background(), t, &Config{Secrets: secrets, Env: env}) tjfi.runTest(context.Background(), t, &Config{Secrets: secrets, Env: env})
} }
func TestRunWithService(t *testing.T) {
if testing.Short() {
t.Skip("skipping integration test")
}
log.SetLevel(log.DebugLevel)
ctx := context.Background()
platforms := map[string]string{
"ubuntu-latest": "node:12.20.1-buster-slim",
}
workflowPath := "services"
eventName := "push"
workdir, err := filepath.Abs("testdata")
assert.NoError(t, err, workflowPath)
runnerConfig := &Config{
Workdir: workdir,
EventName: eventName,
Platforms: platforms,
ReuseContainers: false,
}
runner, err := New(runnerConfig)
assert.NoError(t, err, workflowPath)
planner, err := model.NewWorkflowPlanner(fmt.Sprintf("testdata/%s", workflowPath), true)
assert.NoError(t, err, workflowPath)
plan, err := planner.PlanEvent(eventName)
assert.NoError(t, err, workflowPath)
err = runner.NewPlanExecutor(plan)(ctx)
assert.NoError(t, err, workflowPath)
}
func TestRunActionInputs(t *testing.T) { func TestRunActionInputs(t *testing.T) {
if testing.Short() { if testing.Short() {
t.Skip("skipping integration test") t.Skip("skipping integration test")

26
pkg/runner/testdata/services/push.yaml vendored Normal file
View File

@@ -0,0 +1,26 @@
name: services
on: push
jobs:
services:
name: Reproduction of failing Services interpolation
runs-on: ubuntu-latest
services:
postgres:
image: postgres:12
env:
POSTGRES_USER: runner
POSTGRES_PASSWORD: mysecretdbpass
POSTGRES_DB: mydb
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
steps:
- name: Echo the Postgres service ID / Network / Ports
run: |
echo "id: ${{ job.services.postgres.id }}"
echo "network: ${{ job.services.postgres.network }}"
echo "ports: ${{ job.services.postgres.ports }}"