Clash Royale CLAN TAG

Why does powershell give different result in one-liner than two-liner when converting JSON?

Overview

From a powershell 3 prompt,, I want to call a RESTful service, get some JSON, and pretty-print it. I discovered that if I convert the data to a powershell object, and then convert the powershell object back to json, I get back a nice pretty-printed string. However, if I combine the two conversions into a one-liner with a pipe I will get a different result.

TL;DR: this:

PS> $psobj = $orig | ConvertFrom-JSON
PS> $psobj | ConvertTo-JSON


... gives me different result than this:

PS> $orig | ConvertFrom-JSON | ConvertTo-JSON


Original data

[
{
"Type": "1",
"Name": "QA"
},
{
"Type": "2",
"Name": "whatver"
}
]


Doing the conversion in two steps

I'm going to remove the whitespace (so it fits on one line...), convert it to a powershell object, and then convert it back to JSON. This works well, and gives me back the correct data:

PS> $orig = '[{"Type": "1","Name": "QA"},{"Type": "2","Name": "DEV"}]'
PS> $psobj = $orig | ConvertFrom-JSON
PS> $psobj | ConvertTo-JSON
[
{
"Type": "1",
"Name": "QA"
},
{
"Type": "2",
"Name": "DEV"
}
]


Combining the two steps with a pipe

However, if I combine those last two statements into a one-liner, I get a different result:

PS> $orig | ConvertFrom-JSON | ConvertTo-JSON
{
"value": [
{
"Type": "1",
"Name": "QA"
},
{
"Type": "2",
"Name": "DEV"
}
],
"Count": 2
}


Notice the addition of the keys "value" and "Count". Why is there a difference? I'm sure it has something to do with the desire to return JSON object rather than a JSON array, but I don't understand why the way I do the conversion affects the end result.

4 Answers
4

The solution is to wrap the first two operations with parenthesis:

PS C:> ($orig | ConvertFrom-JSON) | ConvertTo-JSON
[
{
"Type": "1",
"Name": "QA"
},
{
"Type": "2",
"Name": "DEV"
}
]


The parenthesis allow you to grab the output of the first two operations all at once. Without them, powershell will attempt to parse any objects its gets separately. The collection of PSCustomObject resulting from $orig | ConvertFrom-JSON contains two PSCustomObjects for the 1/QA and 2/DEV pairs, so by piping the output of that collection powershell attempts to handle the key/value pairs one-at-a-time.

PSCustomObject

$orig | ConvertFrom-JSON

PSCustomObjects

Using parenthesis is a shorter way of "grouping" that output and allows you to operate on it without making a variable.

ConvertTo-Json

ConvertFrom-Json

1, 2 | ConvertTo-Json

ConvertTo-Json

ConvertTo-Json

{ "value" : [...], "count" : 5 }

First off, why is this happening?

PowerShell automatically wraps multiple objects into a collection called a PSMemberSet that has a Count property on it. It's basically how PowerShell manages arbitrary arrays of objects. What's happening is that the Count property is getting added to the resulting JSON, yielding the undesirable results that you're seeing.

PSMemberSet

Count

Count

We can prove what I just stated above by doing the following:

$Json = @"
[
{
"Type": "1",
"Name": "QA"
},
{
"Type": "2",
"Name": "DEV"
}
]
"@;

# Deserialize the JSON into an array of "PSCustomObject" objects
$Deserialized = ConvertFrom-Json -InputObject $Json;
# Examine the PSBase property of the PowerShell array
# Note the .NET object type name: System.Management.Automation.PSMemberSet
$Deserialized.psbase | Get-Member;


Here is the output from the above

TypeName: System.Management.Automation.PSMemberSet

Name MemberType Definition
---- ---------- ----------
Add Method int IList.Add(System.Object value)
Address Method System.Object&, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 Address(int )
Clear Method void IList.Clear()
......
......
Count Property int Count {get;}


You can work around this behavior by referencing the SyncRoot property of the PSMemberSet (which implements the ICollection .NET interface), and passing the value of that property to ConvertTo-Json.

SyncRoot

PSMemberSet

ICollection

ConvertTo-Json

Here is a complete, working example:

$Json = @"
[
{
"Type": "1",
"Name": "QA"
},
{
"Type": "2",
"Name": "DEV"
}
]
"@;

($Json | ConvertFrom-Json) | ConvertTo-Json;


The correct (expected) output will be displayed, similar to the following:

[
{
"Type": "1",
"Name": "QA"
},
{
"Type": "2",
"Name": "DEV"
}
]
[
{
"Type": "1",
"Name": "QA"
},
{
"Type": "2",
"Name": "DEV"
}
]


($Json | ConvertFrom-Json).SyncRoot

.SyncRoot

Count

psbase

1, 2 | ConvertTo-Json

ConvertTo-Json -InputObject @(1, 2)

ConvertTo-Json

ConvertFrom-Json

.psbase

[System.Management.Automation.PSMemberSet]

.psbase

[object]

.SyncRoot

ICollection

(...)

Note: The problem still exists as of Windows PowerShell v5.1, but PowerShell Core is not affected.

The existing answers provide an effective workaround - enclosing $orig | ConvertFrom-JSON in (...) - but do not explain the problem correctly; also, the workaround cannot be used in all situations.

$orig | ConvertFrom-JSON

(...)

The problematic behavior is a combination of two factors:

ConvertFrom-Json deviates from normal output behavior by sending arrays as single objects through the pipeline. That is, with a JSON string representing an array, ConvertFrom-Json sends the resulting array of objects as a single object through the pipeline.

ConvertFrom-Json

ConvertFrom-Json

You can verify ConvertFrom-Json's surprising behavior as follows:

ConvertFrom-Json

PS> '[ "one", "two" ]' | ConvertFrom-Json | Get-Member

TypeName: System.Object # !! should be: System.String
...


If ConvertFrom-Json passed its output through the pipeline one by one - as cmdlets normally do - Get-Member would instead return the (distinct) types of the items in the collection, which is [System.String] in this case.

ConvertFrom-Json

Get-Member

[System.String]

Enclosing a command in (...) forces enumeration of its output, which is why ($orig | ConvertFrom-Json) | ConvertTo-Json is an effective workaround.

(...)

($orig | ConvertFrom-Json) | ConvertTo-Json

Whether this behavior - which is still present in PowerShell Core too - should be changed is being debated in this GitHub issue.

The System.Array type - the base type for all arrays - has a .Count property defined for it via PowerShell's ETS (Extended Type System - see Get-Help about_Types.ps1xml), which causes ConvertTo-Json to include that property in the JSON string it creates, with the array elements included in a sibling value property.

System.Array

.Count

Get-Help about_Types.ps1xml

ConvertTo-Json

value

This happens only when ConvertTo-Json sees an array as a whole as an input object, as produced by ConvertFrom-Json in this case; e.g., , (1, 2) | ConvertTo-Json surfaces the problem (a nested array whose inner array is sent as a single object), but
1, 2 | ConvertTo-Json does not (the array elements are sent individually).

ConvertTo-Json

ConvertFrom-Json

, (1, 2) | ConvertTo-Json

1, 2 | ConvertTo-Json

This ETS-supplied .Count property was effectively obsoleted in PSv3, when arrays implicitly gained a .Count property due to PowerShell now surfacing explicitly implemented interface members as well, which surfaced the ICollection.Count property (additionally, all objects were given an implicit .Count property in an effort to unify the handling of scalars and collections).

.Count

.Count

ICollection.Count

.Count

Sensibly, this ETS property has therefore been removed in PowerShell Core, but is still present in Windows PowerShell v5.1 - see below for a workaround.

^{Tip of the hat, as many times before, to PetSerAl.}

Note: This workaround is PSv3+ by definition, because the Convert*-Json cmdlets were only introduced in v3.

Convert*-Json

Given that the ETS-supplied .Count property is (a) the cause of the problem and (b) effectively obsolete in PSv3+, the solution is to simply remove it before calling ConvertTo-Json:

.Count

ConvertTo-Json

Remove-TypeData System.Array # Remove the redundant ETS-supplied .Count property


With that, the extraneous .Count and .value properties should have disappeared:

.Count

.value

PS> '[ "one", "two" ]' | ConvertFrom-Json | ConvertTo-Json
[
"one",
"two"
]


The above workaround also fixes the problem for array-valued properties; e.g.:

PS> '' | Select-Object @{ n='prop'; e={ @( 1, 2 ) } } | ConvertTo-Json
{
"prop": [
1,
2
]
}


Without the workaround, the value of "prop" would include the extraneous .Count and .value properties as well.

"prop"

.Count

.value

I've encountered the same problem here. I was able to resolve it by using a ForEach-Object and a PSCustomObject. Hope it helps you.

ForEach-Object

PSCustomObject

'' | ForEach-Object { [PSCustomObject]@{ prop= @( 1, 2 ) }} | ConvertTo-Json


It gives this, which is the expected behavior and using cleaner approach:

{
"prop": [
1,
2
]
}


By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.