yamllint only reports wrong-indentation on the first faulty line

[tamere-allo-peter]

I'm working on a simple tool designed to parse yamllint's output and automatically fix a subset of encountered problems, currently only 11 warning or error conditions are automatically fixed.

It was working just fine until I stumbled upon this problem : yamllint doesn't report all faulty lines with wrong-indentation, only the first one in a block (sort of, I've not done in-depth testing).

$ yamllint --version
yamllint 1.26.3

Consider the following YAML file

---
- name: some task
  xml:
    path: /opt/app/myfile.xml
    xpath: "/blah/{{item}}"
    value: "false"
  with_items:
      - user
      - admin
      - server
  tags: blah

and now consider the output of yamllint with default options and parsable output on this file

$ yamllint -f parsable blah.yml
blah.yml:8:7: [error] wrong indentation: expected 4 but found 6 (indentation)

I believe yamllint should report the same error on line 9 and line 10, but it only reports it on line 8 (the - user line).

Is this the intended behavior or is it a bug ?

Thanks for your time.

[adrienverge]

Hello,

Yes this is intended. You can see a lot of examples of how yamllint is supposed to behave in tests/rules/test_indentation.py, for example: https://github.com/adrienverge/yamllint/blob/d0392b3/tests/rules/test_indentation.py#L692-L700

A way to see this is the first line is wrongly indented, however it's consistent that the second one is indented as the first one.

For your tool, maybe you can parse YAML tokens (especially BlockMappingStart and BlockSequenceStart) to detect all the lines that are subsequently indented?

[tamere-allo-peter]

I understand, thanks for your clarification.

Having an option in yamllint to report all as errors even if they are consistent with the first line would solve my problem, however I don't think I'm able to send a patch for this unfortunately, and I don't even know if this would be easy to do or not :-(

In my tool, which only imports the sys and subprocess modules, I specifically want to avoid having to parse YAML. That's why I let yamllint's parsable output do the job for me, and then all I have to do is mostly to sort warning/errors by line then by column and then insert or delete characters or lines at the right place, based on the line/column reported by yamllint, and offsetting all subsequent errors accordingly.

If an additional option in yamllint is not on the horizon, then I believe I'll modify my tool to fake yamllint's output by creating errors for all subsequent lines until their indentation differs from the first erroneous one.

If I can make it to work, and if I'm allowed to make it public (not sure yet), would you accept such tool as a contribution to yamllint or do you prefer for it to be kept a separate project ?

Have a nice day

[adrienverge]

Thanks for the proposal! In my opinion this goes beyond the scope of yamllint and would be better as a separate project.

[tamere-allo-peter]

@adrienverge ok, thanks anyway. I've been allowed to make it public under the GPLv3+, it's available from https://github.com/opt-nc/yamlfixer in case it might help someone. It's very crude at the moment though, and certainely contains bugs... Thanksfor yamllint !

[schoubi]

hi,

I just discovered your two amazing tools, thanks for that, and i'm stuck at the same point.

I'm trying to lint some salt files, with a lot of values as this :

entity:
    list:
        - value:
           - server1
           - server2
           - server3
        - force: true

As you said, yamllint with yamlfixer will "fix" the file as this, only indenting the first element of the list :

entity:
    list:
        - value:
            - server1
          - server2
          - server3
        - force: true

which in fact broke yaml.
How can i treat this correctly ?

[andrewimeson]

@schoubi it'd be better to discuss that in the yamlfixer project than here.

It's more opinionated (and not configurable) than Jérôme's yamlfixer, but yamlfix can properly transform your first example into valid YAML that passes yamllint

[timansky]

Faced with another bug/feature

  indentation:
    spaces: 2
    indent-sequences: true

dict:
  - item1
    - item2
    - item3

This yaml will pass

[zymurnerd]

Similar to @timansky, I have found what I think is a bug (but I could be missing something intended) with the indent-sequences: false setting. It does appear as though yamllint is only checking the validity of the first item in a sequence, and if it passes, the rest of the block is skipped. Or, at least, the behavior I have seen and tested is consistent with that explanation.

I set my .yamllint to use:

rules:
  indentation:
    spaces: 2
    indent-sequences: false

With this setting I would expect in the example below fruit to fail, veggies to fail, appetizers to pass, and desserts to fail. However, I get the results indicated in the comments in the example.

foods:
  fruit:  # This passes
  - apple
    - orange
    - banana
    - pear
  veggies:  # This passes
  - broccoli
  - carrot
    - pea
    - onion
  appetizers:  # This passes
  - chips
  - salsa
  - salad
  - soup 
  desserts:  # This fails
    - cake
    - pie
    - ice cream
    - cookie

I have tried searching issues to see if this case has been brought up before, and while there are several similar ones, I don't see this case specifically.

Also, I've looked at the indentation test files as pointed to here #427 (comment), but I do not see this test case implemented in the indent-sequences: false cases.

[adrienverge]

Hello @zymurnerd, thanks for these clear details.

However:

• What you describe is different from the original issue. The author of the issue argued that in case of wrong indentation, all lines of the sequence should be reported. For example:

  fruit:
    - apple   # report an error for this line
    - orange  # report another error for this line
    - banana  # again…

• I don't think there is any bug, because the example you provide:

  fruit:
  - apple
    - orange
    - banana

  is valid YAML, but is not a 3-item sequence, it's just a 1-item sequence with value "apple - orange - banana".

  Said differently, this snippet you provided is just another way to write:

  fruit:
  - apple - orange - banana

  or:

  fruit: ["apple - orange - banana"]

Let me know whether this is clear :)

[zymurnerd]

  Said differently, this snippet you provided is just another way to write:
  ```yaml
  fruit:
  - apple - orange - banana
  ```
    
  or:
  ```yaml
  fruit: ["apple - orange - banana"]
  ```

Let me know whether this is clear :)

Yes, this is clear. Thank you for the response. I'm still pretty new to YAML and did not realize that would be a valid, yet different, structure.

Given that this is a valid and different structure that could easily occur as a typo, would it be appropriate for yamllint to provide a warning, perhaps a suppressible one, when these valid indentations occur and indent-sequences is set to false?

[adrienverge]

Given that this is a valid and different structure that could easily occur as a typo, would it be appropriate for yamllint to provide a warning, perhaps a suppressible one, when these valid indentations occur and indent-sequences is set to false?

I understand the idea! But by design, yamllint always tries to report problems with the form, not the content. Here it's really about what the string contains, it would need to analyze text, and if we did that there would be many other examples of structures to report as "potential typos". Also, it would be annoying for users that use "potentially YAML-looking strings", and would get false positive reports.