build,examples: add build system

[EmmaTheMartian]

This PR introduces a small build module into the Vlib, based on the API in Clockwork. It's not quite 1:1 but is still similar.

I am seeking feedback, just leave a comment here if you have any!

This PR will remain a draft while I am receiving feedback, once I feel like it's in a more comfortable state then I will un-draft it.

[huly-for-github]

Connected to Huly®: V_0.6-22257

[phcreery]

I would be nice if a task could be run dependent on the existence of a file or artifact, similar to GNU make. This would be nice for c dependencies and would not run again if the file already exists.

context.artifact(
    name: 'cimgui', 
    file: 'thirdparty/cimgui/libcimgui.a', 
    run: || system('cd thirdparty/cimgui && make static')
)
context.task(
    name: 'build', 
    depends: ['cimgui'], 
    run: || system('v .')
)

[EmmaTheMartian]

I would be nice if a task could be run dependent on the existence of a file or artifact, similar to GNU make. This would be nice for c dependencies and would not run again if the file already exists.

context.artifact(
    name: 'cimgui', 
    file: 'thirdparty/cimgui/libcimgui.a', 
    run: || system('cd thirdparty/cimgui && make static')
)
context.task(
    name: 'build', 
    depends: ['cimgui'], 
    run: || system('v .')
)

You could also just use an if statement in the task's code, like so:

context.task(
        name: 'cimgui', 
        run:  fn () {
                if !exists('thirdparty/cimgui/libcimgui.a') {
                        system('cd thirdparty/cimgui && make static')
                }
        }
)

I do see the benefit in there being a shorthand, although the name artifact feels a little awkward. Maybe that's just because I'm not extremely familiar with Make though :P

[JalonSolov]

object would do just as well. Built objects are "artifacts" of the build system.

[JalonSolov]

Make, for example, checks the timestamp of source files against the timestamp of the built "things", and will not re-build if the built "thing"'s timestamp is later than the source.

For a more concrete example, if compiling C source to .o files, make will follow these steps:

• first check if the .o file exists. If not, the .c file is compiled to the .o.

• if the .o file exists, check the timestamp on the .o and compare to the timestamp on the .c - if the .o timestamp is earlier than the .c, the .c is compiled. The assumption is that the .c must have been modified since the .o was created.

• if the timestamp on the .o is later than the .c, nothing is done. The assumption is that the .c has not been modified since the .o was built, so there is no need to change it.

Most of that won't directly apply to V, unless you want to check the V cache against the V source, but only if -use-cache is on... it isn't the default, yet.

[spytheman]

What happens for just v build.vsh ?
Is there a concept of a default task?

In make, the first rule is the default one.

[EmmaTheMartian]

It errors and tells you that you didn't provide any tasks to run, along with mentioning that passing --tasks will show a list of tasks.

A default task would be good to have, although I'd rather it not just implicitly be the first task. I didn't even know Make done that...

Maybe we could have a default parameter in the params for task(), and when true it will function as the default. If multiple are defined then it would error out with a message saying that there can only be one default task.

[JalonSolov]

I like the idea of a default parameter. make handles it with the .DEFAULT_GOAL special variable, or by you specifying what you want run on the command line.

https://www.gnu.org/software/make/manual/html_node/How-Make-Works.html

Of course, you could still adopt the "first rule is default" style if the default parameter isn't set... just document that's what will happen, and how to set the parameter to change it.

[spytheman]

imho add a v.mod file here too, as well as a common/ folder, with a f1.v and f2.v files in there, each starting with module common, and having 2 different public functions, both used by main.v (after it does import common).

The general idea, is to make it easy to test, that the build system supports dependencies, and imports, so that later, modifying common/f1.v, will lead to invalidating the tasks, that depend on main.v too, and re-running them.

Modifying common/f2.v should do the same.

[spytheman]

Note: v -print-watched-files main.v will output a list of all the .v files, that are involved in the compilation of main.v

[EmmaTheMartian]

The build system is not doing anything to change how import would normally function, nor dependencies. I can add automatic dependency downloading (i.e, automatically running v install to install dependencies in the v.mod file) if that's what you mean

[spytheman]

Build systems are often used for building middle to big projects, that tend to contain several files in them, that depend on each other.

What I mean here, is that the example itself, can contain several files, and I am pointing out the way to do it (adding a v.mod file, + two other, that will be compiled as part of the same project).

That would be useful, so that later, me, or anyone else that wants to review the PR, or check if the build system is suitable for his/her purposes, can check, how the build system reacts to changes to each of the files, without having to create a dummy project from scratch on which to run it, to see the behavior.

[spytheman]

Note: I do not care about the task runner aspect, as much as about the build system aspect.

[EmmaTheMartian]

Build systems are often used for building middle to big projects, that tend to contain several files in them, that depend on each other.

What I mean here, is that the example itself, can contain several files, and I am pointing out the way to do it (adding a v.mod file, + two other, that will be compiled as part of the same project).

That would be useful, so that later, me, or anyone else that wants to review the PR, or check if the build system is suitable for his/her purposes, can check, how the build system reacts to changes to each of the files, without having to create a dummy project from scratch on which to run it, to see the behavior.

Yeah I understand this. Though I'm not sure I see why the build system needs to manage imports at all, if it modified how import would normally behave then I see a reason. At the moment I'm just a little confused by what you mean

[JalonSolov]

He's not saying anything about imports. He's saying that an example for a build system should show examples that make sense for a build system. Only having a single file doesn't really show anything except that the single file can be built.

With multiple files, you can show how if one has already been built, it doesn't get built the next time if it hasn't changed, while those that have been changed are rebuilt, etc.

[spytheman]

If I want to rebuild a project, that has no modifications to its source files, and that was already build, the build system should do nothing (since the target executable will already exist).

If I have a project that has modified files in it, the question of whether the target executable should be rebuild, becomes more complicated, and to be solved, it requires the build system to know/record which .v files were used, for the building of the executable, and to only rebuild it, if at least one of them has been changed.

For example, I may have a project that builds 2 executables, one that is build from tool1.v + common/*.v + somemodule1/*.v, and another one, that is build from tool2.v + common/*.v + somemodule2/*.v .

Note how common/* is used in both, but somemodule1/*.v is used only by tool1.v .

If I make a change in one of the .v files in common/, the executable tool1.exe should be rebuild, even if tool1.v itself was not changed at all. tool2.exe should also be rebuild potentially.

If I make only a change in somemodule1/*.v however, then only tool1.exe should be considered for rebuilding.

[EmmaTheMartian]

Ohhh okay I see what you mean now

[spytheman]

I think that each task may need 2 callbacks: run: and should_run:.

The second callback should_run, could be used by the task runner (if defined), to check, whether the task has been already done (for example by checking if a desired artefact was produced).

Separating the 2 callbacks allows for caching the output of the should_run one, for each build cycle/stage, or calling it multiple times by different threads if needed, assuming that the checks in should_run are relatively cheaper than the work in run, and also idempotent, without risking races of potentially doing the same resource intensive run callback twice in concurrent contexts (imagine for example running the same v -prod file.v command in parallel, each of them overwriting the output of the other one with undefined results).

[spytheman]

A good resource for ideas, may be https://www.cambridge.org/core/services/aop-cambridge-core/content/view/097CE52C750E69BD16B78C318754C7A4/S0956796820000088a.pdf/build_systems_a_la_carte_theory_and_practice.pdf .