Writing and running full e2e integration tests using Go and for Go CLI applications. Lookingo into one of:
@firstname.lastname@example.org or @email@example.com Can I get your thoughts and opinions on any of these three options for testing CLI apps written in Go with the possibility of also measuring coverage – Which means using the test binary under test which I think all three solutions can do? (definitely the first one says it can).
@firstname.lastname@example.org Sorry, haven’t used any of them.
Quickly tried out testcli and it’s a “no go” for me:
- Its README is out-of-date and has an old reference to a package that had its import path changed (easily fixed)
- Running the tests failed miserably as it could not find the
greetingbinary in the
da fuq?! I guess this doesn’t do what I thought – which is to build the test binary and use that to run CLI tests against so you can actually measure coverage 😔
This is eventually what I came up with so far… What do you think @email@example.com ? 🤔
@firstname.lastname@example.org Haven’t used them so far, either, sorry. 🤔
That’s quite a complicated hello world, @email@example.com :-D
main()s often also just do
os.Exit(run()). Passing the command line arguments run
run(…) seems like another obvious thing to do. Even though, I should have done this more consistently, I reckon. I feel very stupid now, because for whatever reason, it never occurred to me to simply pass an
io.Writer for stdout. I really like that. Although, I’m wondering why there’s nothing for stderr. Errors should definitely not mix with other output in my opinion. Anyways. When testing, I always captured stdout with a much more complicated code segment so far. Store the original output and error streams, set the new ones, execute the test, convert things to strings and finally reset to the original streams. I will definitely adopt these
io.Writer arguments. Thanks, mate! :-)
This cmdtest test execution also captures the coverage? It looks kinda more complicated than it should be, otherwise. Just a program with the test definition file would be enough in my opinion.
I don’t know if I like the concept of providing a single test definition file or not. It’s a bit intriguing, but I fear that’s not flexible enough. Just a gut feeling, might be wrong.
@firstname.lastname@example.org Yeah I actually use this technique a lot in GoNix for basically all the Applets. I think this makes it easier to test. The
cmdtest package is kind of cool though really, it basically implements the same kind of test runner as you may (or may not) have seen in the Mercurial test suite. The test files in
testdata are essentially text files that look a bit like you’ve run something on the console and copied pasted them. This is brilliant for e2e cli integration testing 👌 And yes it manages to run the test binary so that coverage can also be measured which is fantastic 👌 – Of course this does not preclude you from writing unit tests for any other parts of your package/library that have a public facing API – But if your public facing API is just the CLI then this is a perfect fit 👌