When do visibility of files and datasets change?#

Files and datasets in LaminDB have the following 3 levels if visibility:

  • 1: “default”, visible in the UI by default, returned from the query results

  • 0: “hidden”, excluded from the query results, notebook files (generated by lamin save) are default to be hidden

  • -1: “trash”, excluded from the query results, set with .delete()

Setup#

!lamin init --storage test-visibility
✅ saved: User(uid='DzTjkKse', handle='testuser1', name='Test User1', updated_at=2023-12-08 11:34:14 UTC)
✅ saved: Storage(uid='WRqGVmpg', root='/home/runner/work/lamindb/lamindb/docs/faq/test-visibility', type='local', updated_at=2023-12-08 11:34:14 UTC, created_by_id=1)
💡 loaded instance: testuser1/test-visibility
💡 did not register local instance on hub

import lamindb as ln
import pandas as pd
💡 lamindb instance: testuser1/test-visibility
file = ln.File(pd.DataFrame({"a": [1, 2], "b": [3, 4]}), description="mydf")
file.save()
❗ no run & transform get linked, consider passing a `run` or calling ln.track()

New file has default visibility 1:

assert file.visibility == 1

When you delete a file, its visibility is set to -1 (“trash”):

file.delete()
❗ moved file to trash
assert file.visibility == -1

Files in trash won’t be returned from default queries:

ln.File.filter(description="mydf").all()
<QuerySet []>

Unless you specify visibility=None to see all hidden and trashed files:

ln.File.filter(description="mydf", visibility=None).all()
<QuerySet [File(uid='o42cLaEH1ZwrjiwY25JV', suffix='.parquet', accessor='DataFrame', description='mydf', size=2240, hash='AdqYk9VZLVMTdfmjwkra9A', hash_type='md5', visibility=-1, key_is_virtual=True, updated_at=2023-12-08 11:34:15 UTC, storage_id=1, created_by_id=1)]>

You can restore a file from trash:

file.restore()
assert file.visibility == 1
ln.File.filter(description="mydf").all()
<QuerySet [File(uid='o42cLaEH1ZwrjiwY25JV', suffix='.parquet', accessor='DataFrame', description='mydf', size=2240, hash='AdqYk9VZLVMTdfmjwkra9A', hash_type='md5', visibility=1, key_is_virtual=True, updated_at=2023-12-08 11:34:15 UTC, storage_id=1, created_by_id=1)]>

Delete test file and instance:

file.delete(permanent=True)
!lamin delete --force test-visibility
💡 deleting instance testuser1/test-visibility
✅     deleted instance settings file: /home/runner/.lamin/instance--testuser1--test-visibility.env
✅     instance cache deleted
✅     deleted '.lndb' sqlite file
❗     consider manually deleting your stored data: /home/runner/work/lamindb/lamindb/docs/faq/test-visibility