View with GROUP_CONCAT returns warnings when using `labkey.selectRows`

[jmtaylor-fhcrc]

The follow R code returns warnings:

library(Rlabkey)

labkey.data <- labkey.selectRows(
    baseUrl="https://dataspace.cavd.org", 
    folderPath="/CAVD", 
    schemaName="CDS", 
    queryName="nabAntigenWithPanelMeta", 
)

This can be isolated to a single column that uses a GROUP_CONCAT SQL operation.

# no warnings
labkey.data <- labkey.selectRows(
  baseUrl="https://dataspace.cavd.org", 
  folderPath="/CAVD", 
  schemaName="CDS",
  colNameOpt="fieldname",
  queryName="nabAntigenWithPanelMeta",
  colSelect = "assay_identifier,antigen_name,antigen_type,virus,virus_type,virus_insert_name,neutralization_tier,clade,antigen_description,antigen_control,virus_full_name,virus_name_other,virus_species,virus_host_cell,virus_backbone,cds_virus_id"
)

# warnings
labkey.data <- labkey.selectRows(
  baseUrl="https://dataspace.cavd.org", 
  folderPath="/CAVD", 
  schemaName="CDS",
  colNameOpt="fieldname",
  queryName="nabAntigenWithPanelMeta",
  colSelect = "panel_names",
)

Several warnings are returned and contain the following information:

1: In listToMatrix(decode$rows, names(tmprow)) :
  subscript out of bounds (index 0 >= vector size 0)

[bbimber]

hey -

I encountered this error in an unrelated situation where I passed a bad filter object (like a value that is a vector, i.e. makeFilter(c('myField', 'IN', c('a', 'b')). It does not sound link this happens here, but I mention it b/c of the same error.

It would be informative to look at the raw JSON that your server produces. You can do this by loading the corresponding dataregion in your browser, and editing the URL to replace 'executeQuery.view' with "selectRows.view".

It might also be useful to try your R code with maxRows=XX and rowOffset to move through your input data and see if there is a specific row or rows that causes this.

[bbimber]

I'm relatively certain this happens when the source field is the result of a groupconcat, perhaps when it's concatenated as CSV, but when the value is empty. I ran into this and I inspected the JSON produced by selectRows, and you will see 'allProject/projects' has values that are JSONArrays. These behave fine. however, the field "Subjects/allProjects/projects" is being reported to the client as an empty array. When I include this field I get the warnings. Without it, I do not.

"rows" : [ {
"Subjects/allProjects/projects" : [ ],
"allProjects/projects" : [ "Malaria 1" ],
"mmr_date" : null,
"CVac1" : null,
"CVac2" : null,
"CVac3" : null,
"sex" : "Female",
"ChallengeType" : null,
"D0" : null,
"groupname" : "Control Animal",
"subjectId" : "30053",
"MMR" : false,
"_labkeyurl_subjectId" : "/list/Labs/Bimber/1172/details.view?name=MalariaAnimals&pk=30053",
"PreExposureDate" : null,

"Protection" : "N\/A",
"ChallengeDate" : "2024-03-29 00:00:00.000"

}

[bbimber]

And I bet this line just needs a check for empty arrays:

labkey-api-r/Rlabkey/src/listToMatrix.cpp

Line 40 in 21c1058

if(!Rf_isNull(gv[0]))

[cnathe]

Sorry for the delay on this, but I just tried querying data from the original use case and I don't see any warnings. What Rlabkey version are you both using? I am on Rlabkey 3.4.2 locally.

[jmtaylor-fhcrc]

I am now using 3.4.2 and I do get the same warnings.

> library(Rlabkey)
Loading required package: httr
Loading required package: jsonlite
> labkey.data <- labkey.selectRows(
  baseUrl="https://dataspace.cavd.org", 
  folderPath="/CAVD", 
  schemaName="CDS",
  colNameOpt="fieldname",
  queryName="nabAntigenWithPanelMeta",
  colSelect = "panel_names",
)
. + There were 50 or more warnings (use warnings() to see the first 50)
> warnings()[1]
Warning message:
In listToMatrix(decode$rows, names(tmprow)) :
  subscript out of bounds (index 0 >= vector size 0)
> sessionInfo()
R version 4.4.3 (2025-02-28)
Platform: x86_64-pc-linux-gnu
Running under: Debian GNU/Linux 12 (bookworm)

...

other attached packages:
[1] Rlabkey_3.4.2  jsonlite_2.0.0 httr_1.4.7    

loaded via a namespace (and not attached):
[1] compiler_4.4.3 R6_2.6.1       tools_4.4.3    curl_6.2.2     Rcpp_1.0.14   

[jmtaylor-fhcrc]

Updating to 3.4.3 on develop has fixed the issue. Thank you!

[bbimber]

Sorry for the delay on this, but I just tried querying data from the original use case and I don't see any warnings. What Rlabkey version are you both using? I am on Rlabkey 3.4.2 locally.

Does what you call the original use case have an empty array in the raw JSON? Because JSON from LabKey with an empty array as the field value will produce this error.

[jmtaylor-fhcrc]

Indeed, the GROUP_CONCAT field is returning empty JSON arrays, in my case.