The secret of $LIST?

Question

Question

Stuart Strickland · Nov 8, 2023

I want to convert a string to uppercase that may or may not be a $LIST so that I can do a case insensitve search. However, when looking at what constitutes a $LIST I can see something that looks like it follows these rules:

Example: S X=$LISTBUILD("hello","world!") gives X=$C(7,1)_"hello"_$C(8,1)_"world!"
The 1st character is $C( how many characters to the start of the next piece)
The 2nd is probably something to do with nested $LISTs
Then you have your raw "hello" followed by $C(8) which is the distance to the end of the next piece
This means that if I have a long string in one of my pieces of between 96 and 121 characters then one of characters inserted by $LIST will be a lowercase alphabetic character. Which in turn means that converting a $LIST to uppercase can break the $LIST and/or return false positives!

To do a case-insensitive search properly each $LIST piece must be converted individually rather than using IF $ZCVT(string, "u") [ $ZCVT(searchFor, "u")

Unless anyone has any better ideas?

Discussion (11)2

Log in or sign up to continue

Vitaliy Serdtsev · Nov 10, 2023

You can convert a list to a string (and vice versa), regardless of the number of nestings. Unfortunately, I can't test this code for Caché 5.x, but I think it should work.
Here is a small example of searching for a string in a list:

#include %systemInclude

 n
 
 s list=$lb(
   "test",
   "for",
   "searching unknown strings here is a very long piece with enough characters to get a lowercase alpha as a $list marker",
   "items",
   "in",
   "aaatArGetwaaaa",
   "lists",
     $lb(
     "/subs",
     "/values",
     "nested list",
     "did you see that ""w"" before the third piece?",
     "Stuart Strickland",
     "Yaron Munz")
 )
 
 s str=$$$UPPER(##class(%Utility).FormatString(list,,.overflow))
 i 'overflow {
   ;s @("LIST="_str) zw LIST
   
   w !,$f(str,$$$UPPER("Targetw"))
 }

Result: 162

2 1

score 0 · Answer 1 · 2023-11-08T07:00:28-05:00

Yaron Munz · Nov 8, 2023

You may use $LISTFIND

$LISTFIND | InterSystems IRIS Data Platform 2023.2

0 0

score 0 · Answer 2 · 2023-11-08T07:31:40-05:00

That would be fine if I was looking for the full element value as my search string. I'm looking for any occurance, full or part, in a lot of data.

From the documentation:
$LISTFIND searches the specified list for the first instance of the requested value. A match must be exact and consist of the full element value. Letter comparisons are case-sensitive. Numbers are compared in canonical form. If an exact match is found, $LISTFIND returns the position of the matching element. If value is not found, $LISTFIND returns a 0.

// Suppose the string I'm looking in is in
S X=$LB("Stuart Strickland","Yaron Munz") // X could be anything
// and I'm looking for "MUNZ"
I $zcvt(X,"u")["MUNZ" Q 1 // I think this is quick but could cause a problem by converting lower case $LIST spacers to upper
S sc=0 F i=1:1:$LL(X) S Y=$LG(X,i) I $zcvt(Y,"u")["MUNZ" s sc=1 // I think this is slow
S sc=$LISTFIND(X,"MUNZ") // I know this will not work
S sc=$LISTFIND($zcvt(X),"MUNZ") // I know this will not work and might even cause an error
Q sc

score 0 · Answer 3 · 2023-11-08T07:30:05-05:00

$list is a binary format, quite complex actually. And you don't need to know this for your scenario

you can use this functions, which may help you with the task

$listvalid - check if the string is a $list, while $listvalid("") is also true

$listlength - counts listitems in $list

$listfind - search for particular item in the presented $list

score 0 · Answer 4 · 2023-11-08T07:44:47-05:00

Thanks Dmitry

I want to find any string in another string that could be a plain string, a $LIST, or a series of nested $LIST to any depth - basically an unkown structure. The string I'm searching in can be any size and the pieces can be any size. And I wan't to do a case insensitive search through a large amount of data and I want instant results. And I want to know the character positions of what was found. And I want the results in a human readable format. And just to make it hard, it has to run on everything from Caché version 5 upwards which means $LISTNEXT, $LISTVALID are not always available.

score 1 · Answer 5 · 2023-11-08T12:56:55-05:00

Could you maybe use $LISTTOSTRING using a delimiter that works for you (something unlikely to be in your actual list data), convert that to uppercase, then use $LISTFROMSTRING to get it back into $LIST format with everything capitalized and not breaking the list?

score 0 · Answer 6 · 2023-11-08T14:05:13-05:00

That's a really good idea. Doesn't quite get there because of nested lists.

Here is a bit more visual info on the problem:

>s x=$LB("test","for","searching unknown strings here is a very long piece with enough characters to get a lowercase alpha as a $list marker","items","in","lists",$lb("/subs","/values","nested list","did you see that ""w"" before the third piece?"))

>f i=1:1:$l(x) s c=$e(x,i) w:c?1c "$c("_$a(c)_")" w:c'?1c c

$c(6)$c(1)test$c(5)$c(1)forw$c(1)searching unknown strings here is a very long piece with enough characters to get a lowercase alpha as a $list marker$c(7)$c(1)items$c(4)$c(1)in$c(7)$c(1)listsM$c(1)$c(7)$c(1)/subs$c(9)$c(1)/values$c(13)$c(1)nested list.$c(1)did you see that "w" before the third piece?

It's those potential lowercase list separators that are forcing me to go the long way round with stepping through each piece. $ListToString may help speed the code up especially as I'm not really expecting deep nesting of lists.

score 0 · Answer 7 · 2023-11-08T16:38:15-05:00

Could you solve this by checking for a $c(1) following the found string?

USER>s target = "Targetw"
USER>s list=$LB("test","falseTarget","searching unknown strings here is a very long piece with enough characters to get a lowercase alpha as a $list marker","items","in","aaatArGetwaaaa","lists",$lb("/subs","/values","nested list","did you see that ""w"" before the third piece?"))
USER>set location=0 f { set location=$find($zcvt(list,"u"),$zcvt(target,"u"),location) quit:location=0 continue:$extract(list,location)=$c(1) quit }
USER>w location
162

score 0 · Answer 8 · 2023-11-09T13:37:51-05:00

Stuart Strickland · Nov 9, 2023

Thank you Erin. That's a nice try but $c(1) isn't guaranteed in a $LIST

s x=$lb(0) zzdump x

0 0

score 0 · Answer 9 · 2023-11-10T12:11:13-05:00

Thanks Vitaliy! That is just what I was looking for!

I suppose that underneath the code, deep down in Quote^%qcr, it may be doing the same as I'm doing, but it is unlikely to have any errors in it and it's not a wild assumption that it will do it in the most efficient way.

Alas, the few version 5 sites will have to live with my slower code

%SYS>w $zv
Cache for UNIX (IBM PowerPC/32-bit) 5.0.18 (Build 6103 + Adhoc 3626) Tue Mar 7 2006 11:55:33 EST
%SYS>W $zcvt(##CLASS(%Library.Utility).FormatString($lb("abc","DEF","",,"tesT",$lb(0))),"u")

W $ZCVT(##CLASS(%Library.Utility).FormatString($LB("abc","DEF","",,"tesT",$LB(0)
^
)),"u")
<CLASS DOES NOT EXIST>
%SYS>W $zcvt(##CLASS(%Utility).FormatString($lb("abc","DEF","",,"tesT",$lb(0))),"u")

W $ZCVT(##CLASS(%Utility).FormatString($LB("abc","DEF","",,"tesT",$LB(0))),"u")
^
<CLASS DOES NOT EXIST>

score 0 · Answer 10 · 2023-11-15T08:57:32-05:00

For versions of Caché 5.0.x, try the following code:

#include %systemInclude

n

s list=$lb(
  "test",
  "for",
  "searching unknown strings here is a very long piece with enough characters to get a lowercase alpha as a $list marker",
  "items",
  "in",
  "aaatArGetwaaaa",
  "lists",
    $lb(
    "/subs",
    "/values",
    "nested list",
    "did you see that ""w"" before the third piece?",
    "Stuart Strickland",
    "Yaron Munz")
)

s str=$$$UPPER($$cccvt^%Wpglo(list,$$$MaxStringLength,.warn))
; or
; s str=$$$UPPER($$listDump^%Wprim(list,9))

i 'warn {
 s str=$e(str,3,$l(str)-2) ; remove << & >>
 ;s @("LIST="_str) zw LIST
 
 w !,$f(str,$$$UPPER("Targetw"))
}

Result: 162

If you end up searching for globals that use $lb(), then you might find it useful:

$$gloFind^%Wprog(gref,string,case,.record) - for Caché 5.0.x
##class(%Global).Find/Get() - for new versions of Caché/IRIS