Query metadata

Query the metadata in the SewerRat backend based on free text, the owner, creation time, etc. This function does not require access to the same filesystem as the SewerRat API.

query(
  text,
  user,
  path,
  from,
  until,
  url,
  number = 100,
  on.truncation = c("message", "warning", "none")
)

formatQueryResults(results)

Arguments

text

String containing a free-text query, see below for details. If missing, no filtering is applied based on the metadata text.

user

String containing the name of the user who generated the metadata. If missing, no filtering is applied based on the user.

path

String containing any component of the path to the metadata file. If missing, no filtering is applied based on the path.

from

A POSIXt object to filter out older files, i.e., only files newer than from will be retained. If missing, no filtering is applied to remove old files.

until

A POSIXt object to filter out newer files, i.e., only files older than until will be retained. If missing, no filtering is applied to remove new files.

url

String containing the URL to the SewerRat REST API.

number

Integer specifying the maximum number of results to return.

on.truncation

String specifying what to do when the number of results exceeds number. Either "warning", "message", or "none".

results

List containing the output of query.

Value

List of lists where each inner list corresponds to a metadata file and contains:

  • path, a string containing the path to the file.

  • user, the identity of the file owner.

  • time, the Unix time of most recent file modification.

  • metadata, a list representing the JSON contents of the file.

For formatQueryResults, a data frame containing path, user, time and metadata. Each row corresponds to one of the search results in results. Each time is now a POSIXct object.

Formatting the text query

For simple use cases, just enter one or more search terms, and the backend search for metadata files that match all the terms.

Advanced users can use the AND and OR keywords to perform binary logical operations. (Make sure to use all-caps for these keywords.) This can be combined with parentheses to control precedence, e.g., (a b OR c d) AND (e f); otherwise, AND takes precedence over OR. Note that any sequence of adjacent search terms are implicitly AND, i.e., the query above can be expanded as ((a AND b) OR (c AND d)) AND (e AND f)).

On a similar note, the NOT keyword can be used for unary negation. This should be put before any search terms, e.g., (NOT a b) AND (c d). If there are no parentheses, any NOT will take precedence over the other boolean operations, i.e., the above query is the same as NOT a b AND c d.

Users can prefix any sequence of search terms with the name of a metadata field, to only search for matches within that field of the metadata file, e.g., (title: prostate cancer) AND (genome: GRCh38 OR genome: GRCm38). Note that this does not extend to the AND, OR and NOT keywords, e.g., title:foo OR bar will not limit the search for bar to the title field.

Users can attach the usual * or ? wildcards to any term to enable pattern matching. For example, neur* will match files with neuron, neural, neurological, etc., while cd? will match files with cd4, cd8, cd9, etc.

Author

Aaron Lun

Examples

# Starting up an example SewerRat service:
info <- startSewerRat()

# Mocking up a directory of stuff to query.
mydir <- tempfile()
dir.create(mydir)
write(file=file.path(mydir, "metadata.json"), '{ "first": "Aaron", "last": "Lun" }')
dir.create(file.path(mydir, "diet"))
write(file=file.path(mydir, "diet", "metadata.json"), 
   '{ "meal": "lunch", "ingredients": "water" }')

# Registering it:
register(mydir, "metadata.json", url=info$url)

query("aaron", url=info$url)
#> [[1]]
#> [[1]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd7aedfe5d/metadata.json"
#> 
#> [[1]]$user
#> [1] "root"
#> 
#> [[1]]$time
#> [1] 1732572190
#> 
#> [[1]]$metadata
#> [[1]]$metadata$first
#> [1] "Aaron"
#> 
#> [[1]]$metadata$last
#> [1] "Lun"
#> 
#> 
#> 
#> [[2]]
#> [[2]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd4a352884/metadata.json"
#> 
#> [[2]]$user
#> [1] "root"
#> 
#> [[2]]$time
#> [1] 1732572190
#> 
#> [[2]]$metadata
#> [[2]]$metadata$first
#> [1] "Aaron"
#> 
#> [[2]]$metadata$last
#> [1] "Lun"
#> 
#> 
#> 
#> [[3]]
#> [[3]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd78e6014/metadata.json"
#> 
#> [[3]]$user
#> [1] "root"
#> 
#> [[3]]$time
#> [1] 1732572190
#> 
#> [[3]]$metadata
#> [[3]]$metadata$first
#> [1] "Aaron"
#> 
#> [[3]]$metadata$last
#> [1] "Lun"
#> 
#> 
#> 
query("lun*", url=info$url) 
#> [[1]]
#> [[1]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd7aedfe5d/metadata.json"
#> 
#> [[1]]$user
#> [1] "root"
#> 
#> [[1]]$time
#> [1] 1732572190
#> 
#> [[1]]$metadata
#> [[1]]$metadata$first
#> [1] "Aaron"
#> 
#> [[1]]$metadata$last
#> [1] "Lun"
#> 
#> 
#> 
#> [[2]]
#> [[2]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd7aedfe5d/diet/metadata.json"
#> 
#> [[2]]$user
#> [1] "root"
#> 
#> [[2]]$time
#> [1] 1732572190
#> 
#> [[2]]$metadata
#> [[2]]$metadata$meal
#> [1] "lunch"
#> 
#> [[2]]$metadata$ingredients
#> [1] "water"
#> 
#> 
#> 
#> [[3]]
#> [[3]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd4a352884/metadata.json"
#> 
#> [[3]]$user
#> [1] "root"
#> 
#> [[3]]$time
#> [1] 1732572190
#> 
#> [[3]]$metadata
#> [[3]]$metadata$first
#> [1] "Aaron"
#> 
#> [[3]]$metadata$last
#> [1] "Lun"
#> 
#> 
#> 
#> [[4]]
#> [[4]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd4a352884/diet/metadata.json"
#> 
#> [[4]]$user
#> [1] "root"
#> 
#> [[4]]$time
#> [1] 1732572190
#> 
#> [[4]]$metadata
#> [[4]]$metadata$meal
#> [1] "lunch"
#> 
#> [[4]]$metadata$ingredients
#> [1] "water"
#> 
#> 
#> 
#> [[5]]
#> [[5]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd78e6014/metadata.json"
#> 
#> [[5]]$user
#> [1] "root"
#> 
#> [[5]]$time
#> [1] 1732572190
#> 
#> [[5]]$metadata
#> [[5]]$metadata$first
#> [1] "Aaron"
#> 
#> [[5]]$metadata$last
#> [1] "Lun"
#> 
#> 
#> 
#> [[6]]
#> [[6]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd78e6014/diet/metadata.json"
#> 
#> [[6]]$user
#> [1] "root"
#> 
#> [[6]]$time
#> [1] 1732572190
#> 
#> [[6]]$metadata
#> [[6]]$metadata$meal
#> [1] "lunch"
#> 
#> [[6]]$metadata$ingredients
#> [1] "water"
#> 
#> 
#> 
query("lun* AND aaron", url=info$url)
#> [[1]]
#> [[1]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd7aedfe5d/metadata.json"
#> 
#> [[1]]$user
#> [1] "root"
#> 
#> [[1]]$time
#> [1] 1732572190
#> 
#> [[1]]$metadata
#> [[1]]$metadata$first
#> [1] "Aaron"
#> 
#> [[1]]$metadata$last
#> [1] "Lun"
#> 
#> 
#> 
#> [[2]]
#> [[2]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd4a352884/metadata.json"
#> 
#> [[2]]$user
#> [1] "root"
#> 
#> [[2]]$time
#> [1] 1732572190
#> 
#> [[2]]$metadata
#> [[2]]$metadata$first
#> [1] "Aaron"
#> 
#> [[2]]$metadata$last
#> [1] "Lun"
#> 
#> 
#> 
#> [[3]]
#> [[3]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd78e6014/metadata.json"
#> 
#> [[3]]$user
#> [1] "root"
#> 
#> [[3]]$time
#> [1] 1732572190
#> 
#> [[3]]$metadata
#> [[3]]$metadata$first
#> [1] "Aaron"
#> 
#> [[3]]$metadata$last
#> [1] "Lun"
#> 
#> 
#> 
query("meal:bar%", url=info$url)
#> list()

query(path="diet/", url=info$url) # has 'diet/' in the path
#> [[1]]
#> [[1]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd7aedfe5d/diet/metadata.json"
#> 
#> [[1]]$user
#> [1] "root"
#> 
#> [[1]]$time
#> [1] 1732572190
#> 
#> [[1]]$metadata
#> [[1]]$metadata$meal
#> [1] "lunch"
#> 
#> [[1]]$metadata$ingredients
#> [1] "water"
#> 
#> 
#> 
#> [[2]]
#> [[2]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd4a352884/diet/metadata.json"
#> 
#> [[2]]$user
#> [1] "root"
#> 
#> [[2]]$time
#> [1] 1732572190
#> 
#> [[2]]$metadata
#> [[2]]$metadata$meal
#> [1] "lunch"
#> 
#> [[2]]$metadata$ingredients
#> [1] "water"
#> 
#> 
#> 
#> [[3]]
#> [[3]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd78e6014/diet/metadata.json"
#> 
#> [[3]]$user
#> [1] "root"
#> 
#> [[3]]$time
#> [1] 1732572190
#> 
#> [[3]]$metadata
#> [[3]]$metadata$meal
#> [1] "lunch"
#> 
#> [[3]]$metadata$ingredients
#> [1] "water"
#> 
#> 
#> 
query(user=Sys.info()["user"], url=info$url) # created by the current user
#> [[1]]
#> [[1]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd7aedfe5d/metadata.json"
#> 
#> [[1]]$user
#> [1] "root"
#> 
#> [[1]]$time
#> [1] 1732572190
#> 
#> [[1]]$metadata
#> [[1]]$metadata$first
#> [1] "Aaron"
#> 
#> [[1]]$metadata$last
#> [1] "Lun"
#> 
#> 
#> 
#> [[2]]
#> [[2]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd7aedfe5d/diet/metadata.json"
#> 
#> [[2]]$user
#> [1] "root"
#> 
#> [[2]]$time
#> [1] 1732572190
#> 
#> [[2]]$metadata
#> [[2]]$metadata$meal
#> [1] "lunch"
#> 
#> [[2]]$metadata$ingredients
#> [1] "water"
#> 
#> 
#> 
#> [[3]]
#> [[3]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd4a352884/metadata.json"
#> 
#> [[3]]$user
#> [1] "root"
#> 
#> [[3]]$time
#> [1] 1732572190
#> 
#> [[3]]$metadata
#> [[3]]$metadata$first
#> [1] "Aaron"
#> 
#> [[3]]$metadata$last
#> [1] "Lun"
#> 
#> 
#> 
#> [[4]]
#> [[4]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd4a352884/diet/metadata.json"
#> 
#> [[4]]$user
#> [1] "root"
#> 
#> [[4]]$time
#> [1] 1732572190
#> 
#> [[4]]$metadata
#> [[4]]$metadata$meal
#> [1] "lunch"
#> 
#> [[4]]$metadata$ingredients
#> [1] "water"
#> 
#> 
#> 
#> [[5]]
#> [[5]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd78e6014/metadata.json"
#> 
#> [[5]]$user
#> [1] "root"
#> 
#> [[5]]$time
#> [1] 1732572190
#> 
#> [[5]]$metadata
#> [[5]]$metadata$first
#> [1] "Aaron"
#> 
#> [[5]]$metadata$last
#> [1] "Lun"
#> 
#> 
#> 
#> [[6]]
#> [[6]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd78e6014/diet/metadata.json"
#> 
#> [[6]]$user
#> [1] "root"
#> 
#> [[6]]$time
#> [1] 1732572190
#> 
#> [[6]]$metadata
#> [[6]]$metadata$meal
#> [1] "lunch"
#> 
#> [[6]]$metadata$ingredients
#> [1] "water"
#> 
#> 
#> 
query(from=Sys.time() - 60, url=info$url) # no more than 1 minute old
#> [[1]]
#> [[1]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd7aedfe5d/metadata.json"
#> 
#> [[1]]$user
#> [1] "root"
#> 
#> [[1]]$time
#> [1] 1732572190
#> 
#> [[1]]$metadata
#> [[1]]$metadata$first
#> [1] "Aaron"
#> 
#> [[1]]$metadata$last
#> [1] "Lun"
#> 
#> 
#> 
#> [[2]]
#> [[2]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd7aedfe5d/diet/metadata.json"
#> 
#> [[2]]$user
#> [1] "root"
#> 
#> [[2]]$time
#> [1] 1732572190
#> 
#> [[2]]$metadata
#> [[2]]$metadata$meal
#> [1] "lunch"
#> 
#> [[2]]$metadata$ingredients
#> [1] "water"
#> 
#> 
#> 
#> [[3]]
#> [[3]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd4a352884/metadata.json"
#> 
#> [[3]]$user
#> [1] "root"
#> 
#> [[3]]$time
#> [1] 1732572190
#> 
#> [[3]]$metadata
#> [[3]]$metadata$first
#> [1] "Aaron"
#> 
#> [[3]]$metadata$last
#> [1] "Lun"
#> 
#> 
#> 
#> [[4]]
#> [[4]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd4a352884/diet/metadata.json"
#> 
#> [[4]]$user
#> [1] "root"
#> 
#> [[4]]$time
#> [1] 1732572190
#> 
#> [[4]]$metadata
#> [[4]]$metadata$meal
#> [1] "lunch"
#> 
#> [[4]]$metadata$ingredients
#> [1] "water"
#> 
#> 
#> 
#> [[5]]
#> [[5]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd78e6014/metadata.json"
#> 
#> [[5]]$user
#> [1] "root"
#> 
#> [[5]]$time
#> [1] 1732572190
#> 
#> [[5]]$metadata
#> [[5]]$metadata$first
#> [1] "Aaron"
#> 
#> [[5]]$metadata$last
#> [1] "Lun"
#> 
#> 
#> 
#> [[6]]
#> [[6]]$path
#> [1] "/tmp/Rtmpf63hzc/file1cd78e6014/diet/metadata.json"
#> 
#> [[6]]$user
#> [1] "root"
#> 
#> [[6]]$time
#> [1] 1732572190
#> 
#> [[6]]$metadata
#> [[6]]$metadata$meal
#> [1] "lunch"
#> 
#> [[6]]$metadata$ingredients
#> [1] "water"
#> 
#> 
#> 

q <- query("lun*", url=info$url)
formatQueryResults(q)
#>                                                 path user                time
#> 1      /tmp/Rtmpf63hzc/file1cd7aedfe5d/metadata.json root 2024-11-25 22:03:10
#> 2 /tmp/Rtmpf63hzc/file1cd7aedfe5d/diet/metadata.json root 2024-11-25 22:03:10
#> 3      /tmp/Rtmpf63hzc/file1cd4a352884/metadata.json root 2024-11-25 22:03:10
#> 4 /tmp/Rtmpf63hzc/file1cd4a352884/diet/metadata.json root 2024-11-25 22:03:10
#> 5       /tmp/Rtmpf63hzc/file1cd78e6014/metadata.json root 2024-11-25 22:03:10
#> 6  /tmp/Rtmpf63hzc/file1cd78e6014/diet/metadata.json root 2024-11-25 22:03:10
#>       metadata
#> 1   Aaron, Lun
#> 2 lunch, water
#> 3   Aaron, Lun
#> 4 lunch, water
#> 5   Aaron, Lun
#> 6 lunch, water