heliport.unicore package
Subpackages
- heliport.unicore.migrations package
- Submodules
- heliport.unicore.migrations.0001_initial module
- heliport.unicore.migrations.0002_unicoreresource_heliport_project module
- heliport.unicore.migrations.0003_dbjob_dbstoragepath_delete_unicoreresource module
- heliport.unicore.migrations.0004_dbpathdir_dbpathfile_delete_dbstoragepath module
- Module contents
Submodules
heliport.unicore.admin module
heliport.unicore.apps module
heliport.unicore.conf module
- class heliport.unicore.conf.UnicoreAppConf(**kwargs)
Bases:
AppConf
Settings of the
heliport.unicore
app.All settings in this class can be overwritten in settings.py
- JOB_TAG_PREFIX = 'heliport-project'
- REST_CORE_URL = 'https://unicore.foo.de/rest/core'
- SERVER_DN = 'CN=unicore.foo.de,O=foo,L=Dresden,ST=Saxony,C=DE'
- SERVER_PUBLIC_KEY = '-----BEGIN PUBLIC KEY-----\r\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAk6Zkq7Pf70brVLhLn6kT\r\n3VWKetArwPsSt9PICQ+XVl3wYQoMWbzr9GVtqKHHDuglSM/skmk3m/I/x8rRZAn0\r\nhdO5V6Aa/vn6+nv7t6KTxnWDlt2pQ4hed6HYTBBYYtIiGz3opTsKcMaOou1kFHHq\r\nhWDqNz/uQgZwa/Om7fNrx7R9zRV77+t88uElcsCzXNwDDSrrKml4zAlJfpoST3k4\r\njwCNO5+SZHEUXwTOJ4TCEbJ0QgS3ohVy9sbdrokJSx9UYZ+DYxWzQp35vnOk/1cJ\r\n9r/g0G+J2y6HoGdCyKe1mQA5ZncNHgw58LVy4gQDIwtgK0DVuMjTZibo+tz5E7ys\r\nOQIDAQAB\r\n-----END PUBLIC KEY-----'
heliport.unicore.decorators module
- heliport.unicore.decorators.require_content_type(content_type)
Decorator factory to ensure a request uses the correct Content-Type.
heliport.unicore.interface module
heliport.unicore.mixins module
- class heliport.unicore.mixins.UnicoreMixin
Bases:
object
Mixin that provides common UNICORE functionality for class-based views.
- Parameters:
unicore_login (heliport.models.LoginInfo) – Login used for interaction with the UNICORE/X server
unicore_transport (pyunicore.client.Transport) – Transport layer for
client
unicore_client (pyunicore.client.Client) – UNICORE client
unicore_user_dn – The DN of the UNICORE user as returned by UNICORE
unicore_job_tag_prefix (str) – Prefix for UNICORE job tags that HELIPORT is supposed to pick up.
- get_context_data(**kwargs)
Create method context.
For more info see
django.views.generic.base.ContextMixin.get_context_data()
.
- setup(request, *args, **kwargs)
Initializes the view.
For more info see
django.views.generic.base.View.setup()
.
- class heliport.unicore.mixins.UnicoreServerAuthenticationRequiredMixin(*args, **kwargs)
Bases:
object
Mixin for views that require authentication from the UNICORE server.
The UNICORE server authenticates by sending a JWT in the Authorization header. This token is signed with the UNICORE server certificate and can be validated using its public key. This mixin reads the claims from the signed token and adds the UNICORE server DN and UNICORE user DN (as given by the issuer and subject claim respectively) to the view.
This mixin should not be used together with the
UnicoreMixin
as both of them try to provide member variables that can be used to interact with a UNICORE server on behalf of the user.- Parameters:
unicore_server_is_authenticated (bool) – Whether the UNICORE server authentication was successful.
unicore_server_dn (Optional[str]) – The DN of the server as given by the “iss” claim.
unicore_user_dn (Optional[str]) – The DN of the UNICORE user as given by the “sub” claim.
unicore_login (
LoginInfo
) – The login of the UNICORE user for whom this request was sentheliport_user (heliport.core.user.HeliportUser) – The HELIPORT user associated with the UNICORE user for whom this request was sent
- dispatch(request, *args, **kwargs)
- setup(request, *args, **kwargs)
heliport.unicore.models module
- class heliport.unicore.models.DBJob(*args, **kwargs)
Bases:
UnicoreResource
,DigitalObject
Django model for UNICORE jobs.
- exception DoesNotExist
Bases:
DoesNotExist
- exception MultipleObjectsReturned
Bases:
MultipleObjectsReturned
- digitalobject_ptr
Accessor to the related object on the forward side of a one-to-one relation.
In the example:
class Restaurant(Model): place = OneToOneField(Place, related_name='restaurant')
Restaurant.place
is aForwardOneToOneDescriptor
instance.
- digitalobject_ptr_id
- job_id
A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.
- resource_url
A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.
- classmethod save_job_list(jobs: Iterable[Job], project: Project | None = None)
Store a list of
Job
s in the database as digital objects.Optionally, if a
Project
is given, add the digital objects to this project.
- unicore_resource_id
A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.
- class heliport.unicore.models.DBPathDir(*args, **kwargs)
Bases:
UnicoreResource
,DigitalObject
,DirectoryObj
Django model for directory in unicore storage.
- exception DoesNotExist
Bases:
DoesNotExist
- exception MultipleObjectsReturned
Bases:
MultipleObjectsReturned
- as_directory(context: Context) Directory
This makes it a directory in HELIPORT. Returns a
UnicoreDirectory
.
- digitalobject_ptr
Accessor to the related object on the forward side of a one-to-one relation.
In the example:
class Restaurant(Model): place = OneToOneField(Place, related_name='restaurant')
Restaurant.place
is aForwardOneToOneDescriptor
instance.
- digitalobject_ptr_id
- path
A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.
- resource_url
A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.
- unicore_resource_id
A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.
- class heliport.unicore.models.DBPathFile(*args, **kwargs)
Bases:
UnicoreResource
,DigitalObject
,FileObj
Django model for file in unicore storage.
- exception DoesNotExist
Bases:
DoesNotExist
- exception MultipleObjectsReturned
Bases:
MultipleObjectsReturned
- as_file(context: Context) File
This makes it a file in HELIPORT. Returns a
UnicoreFile
.
- digitalobject_ptr
Accessor to the related object on the forward side of a one-to-one relation.
In the example:
class Restaurant(Model): place = OneToOneField(Place, related_name='restaurant')
Restaurant.place
is aForwardOneToOneDescriptor
instance.
- digitalobject_ptr_id
- path
A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.
- resource_url
A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.
- unicore_resource_id
A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.
- class heliport.unicore.models.UnicoreClientFor(user: HeliportUser)
Bases:
object
wraps
unicore_client()
in aheliport.core.utils.context.Description
.- generate(context)
- user: HeliportUser
- class heliport.unicore.models.UnicoreDirectory(storage_url: str, path: str, context: Context)
Bases:
UnicorePath
,Directory
The actual directory with
get_parts()
implemented.- get_file_info() Dict[str, str]
Get last access date for the user to see.
- get_parts() Iterable[FileOrDirectory]
Get the contents of this directory as
UnicorePathDirObj
orUnicorePathFileObj
. Ignores hidden files and directories.
- get_path_dir() PathDir
Pyunicore
PathDir
of this directory.
- class heliport.unicore.models.UnicoreFile(storage_url: str, path: str, context: Context)
Bases:
UnicorePath
,File
An actual readable UNICORE file.
- close()
Closes file if open; you can also use
with
block instead of calling this.
- get_file_info() Dict[str, str]
Get the size and last accessed date for the user to see (if available).
- get_path_file() PathFile
Get pyunicore
PathFile
of this object.
- mimetype() str
Guess mimetype from file name.
- size() int | None
File size in bytes.
- class heliport.unicore.models.UnicorePath(storage_url: str, path: str, context: Context)
Bases:
FileOrDirectory
,ABC
Base class for paths inside a UNICORE storage.
- get_client() Client
Get or create a UNICORE client from the current context.
- path: str
The path of this file in the storage returned by
get_storage()
- class heliport.unicore.models.UnicorePathDirObj(storage_url: str, path: str, label: str | None = None)
Bases:
UnicorePathObj
,DirectoryObj
Represent a UNICORE directory that is not in the HELIPORT database.
- as_digital_object(context: Context) DigitalObject
Return this as a
DBPathDir
in the database (create if none exists).
- as_directory(context: Context) Directory
Makes this usable as a directory in HELIPORT; returns
UnicoreDirectory
.
- static resolve_digital_object(storage_url: str, path: str, user: HeliportUser | None) DBPathDir | None
Try to get a
DBPathDir
version of this directory.
- static type_id() str
Used to identify this type during resolution.
- class heliport.unicore.models.UnicorePathFileObj(storage_url: str, path: str, label: str | None = None)
Bases:
UnicorePathObj
,FileObj
Represents a file in unicore storage that is not in the HELIPORT database.
- as_digital_object(context: Context) DigitalObject
Return this as a
DBPathFile
in the database (create if none exists).
- as_file(context: Context) File
Makes this an object representing a file; returns a
UnicoreFile
.
- static resolve_digital_object(storage_url: str, path: str, user: HeliportUser | None) DBPathFile | None
Try to find a matching
DBPAthFile
instance.
- static type_id() str
Used in resolution to identify this type.
- class heliport.unicore.models.UnicorePathObj(storage_url: str, path: str, label: str | None = None)
Bases:
GeneralDigitalObject
,ABC
Abstract base class for files and directory in unicore storages not stored in DB.
- as_html() str
Represent just as string in html.
- as_rdf() RootedRDFGraph
Dummy implementation that represents this as string.
- as_text() str
Represent by label or filename if no label provided. So normally label is unnecessary. But storages itself have no file name (path is “”).
- as_tuple()
For
__hash__
and__eq__
implementation.
- property for_job
Determine if this file or directory is created for a specific UNICORE job.
- get_identifying_params() Dict[str, str]
Parameters which can be used to resolve this instance.
- classmethod resolve(params: Dict[str, str], context: Context) GeneralDigitalObject | None
Resolve an instance by its identifying params (
get_identifying_params()
).Tries to return something in the database before returning something not in DB.
- abstract static resolve_digital_object(storage_url: str, path: str, user: HeliportUser | None) Any | None
Implement this in subclass to handle resolution to object in database.
- class heliport.unicore.models.UnicoreResource(*args, **kwargs)
Bases:
Model
Base class for UNICORE Resources as Digital Objects.
UnicoreResource
can not inherit from DigitalObject because DigitalObjects don’t support subclasses of subclasses yet.- resource_url
A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.
- unicore_resource_id
A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.
- heliport.unicore.models.unicore_client(user: HeliportUser) Client
Create a pyunicore
Client
for the given user.
heliport.unicore.tests module
- class heliport.unicore.tests.StorageTests(methodName='runTest')
Bases:
WithProject
- test_delete()
- test_list()
- class heliport.unicore.tests.TestCallbackView(*args, **kwargs)
Bases:
TestCase
Tests for
unicore.views.CallbackView
.- property default_claims
- path = '/unicore/notify/'
- post_notification(claims=None, path=None, data=None)
POST a notification to the endpoint.
- test_as_by_unicore()
Correct requests as UNICORE would do them should receive HTTP 204.
- test_data_missing_href()
Requests sending data without “href” field should receive HTTP 400.
- test_expired_token()
Requests using an expired token should receive HTTP 403.
- test_malformed_data()
Requests sending malformed JSON data should receive HTTP 400.
- test_missing_jwt_claims()
Requests with missing required JWT claims should receive HTTP 403.
- test_missing_slash()
Posting to the correct path but with the trailing slash missing must work, too.
- test_no_auth()
Requests without Authorization header should receive HTTP 403.
- test_not_post()
All request methods except POST should receive HTTP 405.
CONNECT is not tested as it doesn’t seem to be supported by the test client.
- test_wrong_auth_method()
Requests using the wrong authentication method should receive HTTP 403.
- test_wrong_auth_token()
Requests using a token which can’t be decoded should receive HTTP 403.
- test_wrong_content_type()
Requests with the wrong Content-Type should receive HTTP 415.
heliport.unicore.urls module
heliport.unicore.views module
- class heliport.unicore.views.CallbackView(*args, **kwargs)
Bases:
UnicoreServerAuthenticationRequiredMixin
,View
Can be called by UNICORE to notify HELIPORT about a job’s status.
- dispatch(request, *args, **kwargs)
- post(request, *args, **kwargs)
Method called on POST request.
For more info see
django.views.generic.base.View.dispatch()
.This method is can be called by a UNICORE instance to give updates about a job’s status. If the job can be associated with a user and a project, it is imported into the HELIPORT database. The user that owns the job is identified by the DN stored in the data field of their “UNICORE”
LoginInfo
. The project is identified by the job tags. The job is only imported and added to the project if the identified user is a member of said project.
- class heliport.unicore.views.JobActionView
Bases:
UnicoreMixin
,View
View that allows the user to perform some actions on a UNICORE job.
- Parameters:
allowed_actions (List[str]) – The actions that can be performed on a job
- get(request, job_id, action)
Method called on GET request.
For more info see
django.views.generic.base.View.dispatch()
.
- class heliport.unicore.views.JobDetailView
Bases:
HeliportObjectMixin
,UnicoreMixin
,DetailView
- get_context_data(**kwargs)
- pk_url_kwarg = 'job'
- template_name = 'unicore/job.html'
- class heliport.unicore.views.JobsView
Bases:
HeliportProjectMixin
,UnicoreMixin
,TemplateView
List of the UNICORE jobs that belong to a project.
- Parameters:
unicore_job_tag_project (str) – Jobs with this tag are automatically associated with this project.
- get_context_data(**kwargs)
Create method context.
For more info see
django.views.generic.base.ContextMixin.get_context_data()
.
- setup(request, *args, **kwargs)
- template_name = 'unicore/jobs.html'
- class heliport.unicore.views.StorageView
Bases:
HeliportProjectMixin
,UnicoreMixin
,DetailView
shows a list of storages and allows creation and editing.
- from_url(url, context)
Create pyunicore
Storage
from url, reusing existing client in context.
- get_context_data(**kwargs)
Called by Django to get rendering context; get files and directories.
- label_for(storage)
Return a string label to present the
storage
.
- pk_url_kwarg = 'project'
- post(request, *args, **kwargs)
Handle post request; add or delete a storage.
- template_name = 'unicore/storages.html'
- class heliport.unicore.views.TestJobView
Bases:
UnicoreMixin
,View
View that, on behalf of the user, sends a job to the cluster.
This is meant for testing purposes. The job runs on the login node and executes the following code:
echo Hello World
The job will be tagged so that it can be found in the appropriate HELIPORT project.
- get(request)
Method called on GET request.
For more info see
django.views.generic.base.View.dispatch()
.