Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build,examples: add build system #23853

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions examples/build_system/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Files and directories made by build.vsh:
/target/
/test.txt

# Pre-compiled build.vsh
/build
58 changes: 58 additions & 0 deletions examples/build_system/build.vsh
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
#!/usr/bin/env -S v run

import build
import time

// Define variables that can be used to change tasks in the buildscript
const app_name = 'hello'
const program_args = 'World'
const build_dir = 'target'

// Make the build context
mut context := build.context(
// Set the default task to `release` when no arguments are provided
default: 'release'
)

// Add a few simple tasks
context.task(name: 'doc', run: |self| system('echo "Nothing to do"'))
context.task(name: 'run', run: |self| system('v run . ${program_args}'))
context.task(name: 'build', run: |self| system('v .'))
context.task(name: 'build.prod', run: |self| system('v -prod -o ${app_name} .'))

// `_` to denote "private" tasks. Nothing stops the user from using it, but
// this tells them that the task is not meant to be used by them.
context.task(
name: '_mkdirs'
// The `help` field is displayed in `--tasks` to give a short summary of what the task does.
help: 'Makes the directories used by the application'
run: fn (self build.Task) ! {
if !exists(build_dir) {
mkdir_all(build_dir) or { panic(err) }
}
}
)

// This task will only run when the `test.txt` file is outdated
context.artifact(
name: 'test.txt'
help: 'Generate test.txt'
run: fn (self build.Task) ! {
write_file('test.txt', time.now().str())!
}
)

// Add a more complex task
context.task(
name: 'release'
help: 'Build the app in production mode, generates documentation, and releases the build on Git'
depends: ['_mkdirs', 'doc', 'test.txt']
run: fn (self build.Task) ! {
system('v -prod -o ${build_dir}/${app_name} .')
// Pretend we are using Git to publish the built file as a release here.
}
)

// Run the build context. This will iterate over os.args and each corresponding
// task, skipping any arguments that start with a hyphen (-)
context.run()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens for just v build.vsh ?
Is there a concept of a default task?

In make, the first rule is the default one.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It errors and tells you that you didn't provide any tasks to run, along with mentioning that passing --tasks will show a list of tasks.

A default task would be good to have, although I'd rather it not just implicitly be the first task. I didn't even know Make done that...

Maybe we could have a default parameter in the params for task(), and when true it will function as the default. If multiple are defined then it would error out with a message saying that there can only be one default task.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea of a default parameter. make handles it with the .DEFAULT_GOAL special variable, or by you specifying what you want run on the command line.

https://www.gnu.org/software/make/manual/html_node/How-Make-Works.html

Of course, you could still adopt the "first rule is default" style if the default parameter isn't set... just document that's what will happen, and how to set the parameter to change it.

5 changes: 5 additions & 0 deletions examples/build_system/main.v
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
import os
Copy link
Member

@spytheman spytheman Mar 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imho add a v.mod file here too, as well as a common/ folder, with a f1.v and f2.v files in there, each starting with module common, and having 2 different public functions, both used by main.v (after it does import common).

The general idea, is to make it easy to test, that the build system supports dependencies, and imports, so that later, modifying common/f1.v, will lead to invalidating the tasks, that depend on main.v too, and re-running them.

Modifying common/f2.v should do the same.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: v -print-watched-files main.v will output a list of all the .v files, that are involved in the compilation of main.v

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The build system is not doing anything to change how import would normally function, nor dependencies. I can add automatic dependency downloading (i.e, automatically running v install to install dependencies in the v.mod file) if that's what you mean

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build systems are often used for building middle to big projects, that tend to contain several files in them, that depend on each other.

What I mean here, is that the example itself, can contain several files, and I am pointing out the way to do it (adding a v.mod file, + two other, that will be compiled as part of the same project).

That would be useful, so that later, me, or anyone else that wants to review the PR, or check if the build system is suitable for his/her purposes, can check, how the build system reacts to changes to each of the files, without having to create a dummy project from scratch on which to run it, to see the behavior.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: I do not care about the task runner aspect, as much as about the build system aspect.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build systems are often used for building middle to big projects, that tend to contain several files in them, that depend on each other.

What I mean here, is that the example itself, can contain several files, and I am pointing out the way to do it (adding a v.mod file, + two other, that will be compiled as part of the same project).

That would be useful, so that later, me, or anyone else that wants to review the PR, or check if the build system is suitable for his/her purposes, can check, how the build system reacts to changes to each of the files, without having to create a dummy project from scratch on which to run it, to see the behavior.

Yeah I understand this. Though I'm not sure I see why the build system needs to manage imports at all, if it modified how import would normally behave then I see a reason. At the moment I'm just a little confused by what you mean

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

He's not saying anything about imports. He's saying that an example for a build system should show examples that make sense for a build system. Only having a single file doesn't really show anything except that the single file can be built.

With multiple files, you can show how if one has already been built, it doesn't get built the next time if it hasn't changed, while those that have been changed are rebuilt, etc.

Copy link
Member

@spytheman spytheman Mar 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I want to rebuild a project, that has no modifications to its source files, and that was already build, the build system should do nothing (since the target executable will already exist).

If I have a project that has modified files in it, the question of whether the target executable should be rebuild, becomes more complicated, and to be solved, it requires the build system to know/record which .v files were used, for the building of the executable, and to only rebuild it, if at least one of them has been changed.

For example, I may have a project that builds 2 executables, one that is build from tool1.v + common/*.v + somemodule1/*.v, and another one, that is build from tool2.v + common/*.v + somemodule2/*.v .

Note how common/* is used in both, but somemodule1/*.v is used only by tool1.v .

If I make a change in one of the .v files in common/, the executable tool1.exe should be rebuild, even if tool1.v itself was not changed at all. tool2.exe should also be rebuild potentially.

If I make only a change in somemodule1/*.v however, then only tool1.exe should be considered for rebuilding.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ohhh okay I see what you mean now


fn main() {
println('Hello, ${os.args[1]}!')
}
52 changes: 52 additions & 0 deletions vlib/build/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
## Description

`build` provides a small build system leveraging V(SH) for the buildscript.

## Example

> See also: [build_system example](https://github.com/vlang/v/tree/master/examples/build_system)

```v
#!/usr/bin/env -S v run

import build
// .vsh automatically imports `os`, so you don't need this typically
import os { system }

const app_name = 'vlang'
const program_args = 'World'

mut context := build.context(
// Set the default task to `release` when no arguments are provided
default: 'release'
)

context.task(name: 'doc', run: |self| system('v doc .'))
context.task(name: 'run', run: |self| system('v run . ${program_args}'))
context.task(name: 'build', run: |self| system('v .'))
context.task(name: 'build.prod', run: |self| system('v -prod .'))

context.task(
name: 'release'
depends: ['doc']
run: fn (self build.Task) ! {
system('v -prod -o build/${app_name} .')
// You could use Git to publish a release here too
}
)

context.run()
```

## Pre-Compiling

Running VSH scripts requires V to compile the script before executing it, which can cause a delay
between when you run `./build.vsh` and when the script actually starts executing.

If you want to fix this, you can "pre-compile" the buildscript by building the script, i.e, running
`v -skip-running build.vsh`.

> You will need to rebuild every time you change the buildscript, and you should also add `/build`
> to your `.gitignore`

> If you want maximum speed, you can also `v -prod -skip-running build.vsh`
180 changes: 180 additions & 0 deletions vlib/build/build.v
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
module build

import os

@[heap; noinit]
pub struct BuildContext {
mut:
// should_run caches the result of should_run from tasks.
should_run map[string]bool
tasks []Task
pub mut:
// default is the default task to run when no others are provided.
default ?string
}

@[heap; noinit]
pub struct Task {
run fn (Task) ! @[required]
should_run fn (Task) !bool @[required]
// repeatable controls whether or not this task can run multiple times per build cycle
repeatable bool
pub:
name string
help string
depends []string
mut:
did_run bool
}

@[params]
pub struct BuildContextParams {
pub:
default ?string
}

@[params]
pub struct TaskParams {
pub:
name string @[required]
help string
depends []string
should_run fn (Task) !bool = |self| true
run fn (Task) ! @[required]
// repeatable controls whether or not this task can run multiple times per build cycle
repeatable bool
}

@[params]
pub struct ArtifactParams {
pub:
name string @[required]
help string
depends []string
should_run fn (Task) !bool = |self| !os.exists(self.name)
run fn (Task) ! @[required]
// repeatable controls whether or not this task can run multiple times per build cycle
repeatable bool
}

// context creates an empty BuildContext.
pub fn context(params BuildContextParams) BuildContext {
return BuildContext{
default: params.default
}
}

// task creates a task for the given context.
pub fn (mut context BuildContext) task(config TaskParams) {
if context.get_task(config.name) != none {
eprintln('error: task already exists with name `${config.name}`')
exit(1)
}
context.tasks << Task{
should_run: config.should_run
run: config.run
name: config.name
help: config.help
depends: config.depends
}
}

// artifact creates an artifact task for the given context.
pub fn (mut context BuildContext) artifact(config ArtifactParams) {
if context.get_task(config.name) != none {
eprintln('error: task already exists with name `${config.name}`')
exit(1)
}
context.tasks << Task{
should_run: config.should_run
run: config.run
name: config.name
help: config.help
depends: config.depends
repeatable: config.repeatable
}
}

// get_task gets the task with the given name.
pub fn (mut context BuildContext) get_task(name string) ?&Task {
for mut task in context.tasks {
if task.name == name {
return mut task
}
}
return none
}

// exec executes the task with the given name in the context.
pub fn (mut context BuildContext) exec(name string) {
if mut task := context.get_task(name) {
task.exec(mut context)
} else {
eprintln('error: no such task: ${name}')
exit(1)
}
}

// exec runs the given task and its dependencies
pub fn (mut task Task) exec(mut context BuildContext) {
if task.did_run && !task.repeatable {
println(': ${task.name} (skipped)')
return
}

if task.name !in context.should_run {
context.should_run[task.name] = task.should_run(task) or {
eprintln('error: failed to call should_run for task `${task.name}`: ${err}')
exit(1)
}
}

if !context.should_run[task.name] {
println(': ${task.name} (skipped)')
return
}

for dep in task.depends {
if dep == task.name {
eprintln('error: cyclic task dependency detected, `${task.name}` depends on itself')
exit(1)
}

context.exec(dep)
}
println(': ${task.name}')
task.did_run = true
task.run(task) or {
eprintln('error: failed to run task `${task.name}`: ${err}')
exit(1)
}
}

// run executes all tasks provided through os.args.
pub fn (mut context BuildContext) run() {
// filter out options
mut tasks := os.args[1..].filter(|it| !it.starts_with('-'))

// check options
if '--tasks' in os.args || '-tasks' in os.args {
println('Tasks:')
for _, task in context.tasks {
println('- ${task.name}: ${task.help}')
}
return
}

if tasks.len == 0 {
if context.default != none {
tasks << context.default
} else {
eprintln('error: no task provided, run with `--tasks` for a list')
exit(1)
}
}

// execute tasks
for arg in tasks {
context.exec(arg)
}
}
Loading