Odtnhj

I used a bazel macro to run a python test on a subset of source files. Similar to this:

def report(name, srcs):
 source_labels = [file for file in srcs if file.startswith("a")]
 if len(source_labels) == 0:
 return;

 source_filenames = ["$(location %s)" % x for x in source_labels]

 native.py_test(
 name = name + "_report",
 srcs = ["report_tool"],
 data = source_labels,
 main = "report_tool.py",
 args = source_filenames,
 )

report("foo", ["foo.hpp", "afoo.hpp"])

This worked fine until one of my source files started using a select and now I get the error:

File "/home/david/foo/report.bzl", line 47, in report
 [file for file in srcs if file.startswith("a")]
type 'select' is not iterable

I tried to move the code to a bazel rule, but then I get a different error that py_test can not be used in the analysis phase.

The reason that the select is causing the error is that macros are evaluated during the loading phase, whereas selectss are not evaluated until the analysis phase (see Extension Overview).

Similarly, py_test can't be used in a rule implementation because the rule implementation is evaluated in the analysis phase, whereas the py_test would need to have been loaded in the loading phase.

One way past this is to create a separate Starlark rule that takes a list of labels and just creates a file with each filename from the label. Then the py_test takes that file as data and loads the other files from there. Something like this:

def report(name, srcs):

 file_locations_label = "_" + name + "_file_locations"
 _generate_file_locations(
 name = file_locations_label,
 labels = srcs
 )

 native.py_test(
 name = name + "_report",
 srcs = ["report_tool.py"],
 data = srcs + [file_locations_label],
 main = "report_tool.py",
 args = ["$(location %s)" % file_locations_label]
 )


def _generate_file_locations_impl(ctx):
 paths = 
 for l in ctx.attr.labels:
 f = l.files.to_list()[0]
 if f.basename.startswith("a"):
 paths.append(f.short_path)
 ctx.actions.write(ctx.outputs.file_paths, "n".join(paths))
 return DefaultInfo(runfiles = ctx.runfiles(files = [ctx.outputs.file_paths]))

_generate_file_locations = rule(
 implementation = _generate_file_locations_impl,
 attrs = "labels": attr.label_list(allow_files = True) ,
 outputs = "file_paths": "%name_files" ,
)

This has one disadvantage: Because the py_test has to depend on all the sources, the py_test will get rerun even if the only files that have changed are the ignored files. (If this is a significant drawback, then there is at least one way around this, which is to have _generate_file_locations filter the files too, and have the py_test depend on only _generate_file_locations. This could maybe be accomplished through runfiles symlinks)

Update:

Since the test report tool comes from an external repository and can't be easily modified, here's another approach that might work better. Rather than create a rule that creates a params file (a file containing the paths to process) as above, the Starlark rule can itself be a test rule that uses the report tool as the test executable:

def _report_test_impl(ctx):
 filtered_srcs = 
 for f in ctx.attr.srcs:
 f = f.files.to_list()[0]
 if f.basename.startswith("a"):
 filtered_srcs.append(f)

 report_tool = ctx.attr._report_test_tool
 ctx.actions.write(
 output = ctx.outputs.executable,
 content = "report_tool paths".format(
 report_tool = report_tool.files_to_run.executable.short_path,
 paths = " ".join([f.short_path for f in filtered_srcs]))
 )

 runfiles = ctx.runfiles(files = filtered_srcs).merge(
 report_tool.default_runfiles)
 return DefaultInfo(runfiles = runfiles)

report_test = rule(
 implementation = _report_test_impl,
 attrs = 
 "srcs": attr.label_list(allow_files = True),
 "_report_test_tool": attr.label(default="//:report_test_tool"),
 ,
 test = True,
)

This requires that the test report tool be a py_binary somewhere so that the test rule above can depend on it:

py_binary(
 name = "report_test_tool",
 srcs = ["report_tool.py"],
 main = "report_tool.py",
)