Odtnhj

I read data from a json file with jq. I wanna append the results into a yaml file, but I dont get it working. I am quite new to shell programming. My goal is to append that "users" to an existing "users"-Array in a yaml file.

This is my json file:

#$DEFAULTS_FILE

"users":
 [
 "name":"pi",
 "gecos": "Hypriot Pirate",
 "sudo":"ALL=(ALL) NOPASSWD:ALL",
 "shell": "/bin/bash",
 "groups":"users,docker,video",
 "plain_text_passwd":"pi",
 "lock_passwd":"false",
 "ssh_pwauth":"true",
 "chpasswd": "expire": false
 ,
 "name":"admin",
 "gecos": "Hypriot Pirate",
 "sudo":"ALL=(ALL) NOPASSWD:ALL",
 "shell": "/bin/bash",
 "primary-group": "users",
 "groups":"users,docker,adm,dialout,audio,plugdev,netdev,video",
 "ssh-import-id":"None",
 "plain_text_passwd":"pi",
 "lock_passwd":"true",
 "ssh_pwauth":"true",
 "chpasswd": "expire: false",
 "ssh-authorized-keys": ["ssh-rsa abcdefg1234567890 YOUR_KEY@YOURHOST.local"]
 
 ]
 

I filter it with that:

cat $DEFAULTS_FILE | jq .users

I have no clue how to convert that json into a yaml.

My expected result should be:

users:
 - name: pi
 gecos: "Hypriot Pirate"
 sudo: ALL=(ALL) NOPASSWD:ALL
 shell: /bin/bash
 groups: users,docker,video
 plain_text_passwd: pi
 lock_passwd: false
 ssh_pwauth: true
 chpasswd: expire: false 
 - name: admin
 primary-group: users
 shell: /bin/bash
 sudo: ALL=(ALL) NOPASSWD:ALL
 groups: users,docker,adm,dialout,audio,plugdev,netdev,video
 ssh-import-id: None

I tried to use a second tool called yq which is similar to jq and can write yaml files. But I have no positive progress.

EDIT

I know that I can add content to the yaml with that:

yq w -i "my.yml" "users[+]" "some content"

But I dont know how to merge my json into that.

Any help or hint would be nice, thank you in advance...

I'm not sure what rules you're using to get to your expected result. It seems like you're randomly applying different rules to how the values are being converted.

As I understand it, scalar values are just output as is (with potential encoding), objects are output as key/value pairs, and array objects are output with a - for every item. The indentation associates what's part of what.

So based on those rules if you're going to use jq:

def yamlify:
 (objects | to_entries | (.value | type) as $type |
 if $type == "array" then
 "(.key):", (.value | yamlify)
 elif $type == "object" then
 "(.key):", " (.value | yamlify)"
 else
 "(.key):t(.value)"
 end
 )
 // (arrays | select(length > 0) | [yamlify] |
 " - (.[0])", " (.[1:])"
 )
 // .
 ;

Then to use it, add it to your .jq file and use it:

$ jq -r yamlify input.json
users:
 - name: pi
 gecos: Hypriot Pirate
 sudo: ALL=(ALL) NOPASSWD:ALL
 shell: /bin/bash
 groups: users,docker,video
 plain_text_passwd: pi
 lock_passwd: false
 ssh_pwauth: true
 chpasswd:
 expire: false
 - name: admin
 gecos: Hypriot Pirate
 sudo: ALL=(ALL) NOPASSWD:ALL
 shell: /bin/bash
 primary-group: users
 groups: users,docker,adm,dialout,audio,plugdev,netdev,video
 ssh-import-id: None
 plain_text_passwd: pi
 lock_passwd: true
 ssh_pwauth: true
 chpasswd: expire: false
 ssh-authorized-keys:
 - ssh-rsa abcdefg1234567890 YOUR_KEY@YOURHOST.local

Here's another variation that aligns the values

def yamlify2:
 (objects | to_entries | (map(.key | length) | max + 2) as $w |
 . | (.value | type) as $type |
 if $type == "array" then
 "(.key):", (.value | yamlify2)
 elif $type == "object" then
 "(.key):", " (.value | yamlify2)"
 else
 "(.key):(" " * (.key | $w - length))(.value)"
 end
 )
 // (arrays | select(length > 0) | [yamlify2] |
 " - (.[0])", " (.[1:])"
 )
 // .
 ;

$ jq -r yamlify2 input.json
users:
 - name: pi
 gecos: Hypriot Pirate
 sudo: ALL=(ALL) NOPASSWD:ALL
 shell: /bin/bash
 groups: users,docker,video
 plain_text_passwd: pi
 lock_passwd: false
 ssh_pwauth: true
 chpasswd:
 expire: false
 - name: admin
 gecos: Hypriot Pirate
 sudo: ALL=(ALL) NOPASSWD:ALL
 shell: /bin/bash
 primary-group: users
 groups: users,docker,adm,dialout,audio,plugdev,netdev,video
 ssh-import-id: None
 plain_text_passwd: pi
 lock_passwd: true
 ssh_pwauth: true
 chpasswd: expire: false
 ssh-authorized-keys:
 - ssh-rsa abcdefg1234567890 YOUR_KEY@YOURHOST.local

I ended up installing a gem and using ruby:

gem install deep_merge (https://github.com/danielsdeleo/deep_merge)

Here is my approach:

#!/usr/bin/env ruby
#
require 'json'
require 'yaml'
require 'deep_merge/rails_compat'

json_input_file = ARGF.argv[0]
yaml_output_file = ARGF.argv[1]
scope = ARGF.argv[2]

json = File.read(json_input_file)
yaml = File.read(yaml_output_file)

json_data = JSON.parse(json)
scoped_result = json_data[scope]

old_values_hash = YAML.load(File.read(yaml_output_file))
result = YAML.parse(yaml)
merged = old_values_hash.deeper_merge scoped_result

File.write(yaml_output_file, merged.to_yaml)

It then can be used to read a specific scoped_value from a json file.

jsonyaml.sh $JSON_FILE $YAML_TARGET "my_scope_to_add"