heliport.unicore package

Subpackages

Submodules

heliport.unicore.admin module

heliport.unicore.apps module

class heliport.unicore.apps.UnicoreConfig(app_name, app_module)

Bases: AppConfig

default_auto_field = 'django.db.models.BigAutoField'
name = 'heliport.unicore'
ready()

heliport.unicore.conf module

class heliport.unicore.conf.UnicoreAppConf(**kwargs)

Bases: AppConf

Settings of the heliport.unicore app.

All settings in this class can be overwritten in settings.py

JOB_TAG_PREFIX = 'heliport-project'
class Meta

Bases: object

prefix = 'HELIPORT_UNICORE'
REST_CORE_URL = 'https://unicore.foo.de/rest/core'
SERVER_DN = 'CN=unicore.foo.de,O=foo,L=Dresden,ST=Saxony,C=DE'
SERVER_PUBLIC_KEY = '-----BEGIN PUBLIC KEY-----\r\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAk6Zkq7Pf70brVLhLn6kT\r\n3VWKetArwPsSt9PICQ+XVl3wYQoMWbzr9GVtqKHHDuglSM/skmk3m/I/x8rRZAn0\r\nhdO5V6Aa/vn6+nv7t6KTxnWDlt2pQ4hed6HYTBBYYtIiGz3opTsKcMaOou1kFHHq\r\nhWDqNz/uQgZwa/Om7fNrx7R9zRV77+t88uElcsCzXNwDDSrrKml4zAlJfpoST3k4\r\njwCNO5+SZHEUXwTOJ4TCEbJ0QgS3ohVy9sbdrokJSx9UYZ+DYxWzQp35vnOk/1cJ\r\n9r/g0G+J2y6HoGdCyKe1mQA5ZncNHgw58LVy4gQDIwtgK0DVuMjTZibo+tz5E7ys\r\nOQIDAQAB\r\n-----END PUBLIC KEY-----'

heliport.unicore.decorators module

heliport.unicore.decorators.require_content_type(content_type)

Decorator factory to ensure a request uses the correct Content-Type.

heliport.unicore.interface module

class heliport.unicore.interface.UnicoreJobs

Bases: Module

get_url(project)
is_configured(project)
module_id = 'unicore_jobs'
name = 'UNICORE Jobs'
class heliport.unicore.interface.UnicoreStorages

Bases: Module

get_url(project)
is_configured(project)
module_id = 'unicore_storages'
name = 'UNICORE Storages'

heliport.unicore.mixins module

class heliport.unicore.mixins.UnicoreMixin

Bases: object

Mixin that provides common UNICORE functionality for class-based views.

Parameters:
  • unicore_login (heliport.models.LoginInfo) – Login used for interaction with the UNICORE/X server

  • unicore_transport (pyunicore.client.Transport) – Transport layer for client

  • unicore_client (pyunicore.client.Client) – UNICORE client

  • unicore_user_dn – The DN of the UNICORE user as returned by UNICORE

  • unicore_job_tag_prefix (str) – Prefix for UNICORE job tags that HELIPORT is supposed to pick up.

get_context_data(**kwargs)

Create method context.

For more info see django.views.generic.base.ContextMixin.get_context_data().

setup(request, *args, **kwargs)

Initializes the view.

For more info see django.views.generic.base.View.setup().

class heliport.unicore.mixins.UnicoreServerAuthenticationRequiredMixin(*args, **kwargs)

Bases: object

Mixin for views that require authentication from the UNICORE server.

The UNICORE server authenticates by sending a JWT in the Authorization header. This token is signed with the UNICORE server certificate and can be validated using its public key. This mixin reads the claims from the signed token and adds the UNICORE server DN and UNICORE user DN (as given by the issuer and subject claim respectively) to the view.

This mixin should not be used together with the UnicoreMixin as both of them try to provide member variables that can be used to interact with a UNICORE server on behalf of the user.

Parameters:
  • unicore_server_is_authenticated (bool) – Whether the UNICORE server authentication was successful.

  • unicore_server_dn (Optional[str]) – The DN of the server as given by the “iss” claim.

  • unicore_user_dn (Optional[str]) – The DN of the UNICORE user as given by the “sub” claim.

  • unicore_login (LoginInfo) – The login of the UNICORE user for whom this request was sent

  • heliport_user (heliport.core.user.HeliportUser) – The HELIPORT user associated with the UNICORE user for whom this request was sent

dispatch(request, *args, **kwargs)
setup(request, *args, **kwargs)

heliport.unicore.models module

class heliport.unicore.models.DBJob(*args, **kwargs)

Bases: UnicoreResource, DigitalObject

Django model for UNICORE jobs.

exception DoesNotExist

Bases: DoesNotExist

exception MultipleObjectsReturned

Bases: MultipleObjectsReturned

digitalobject_ptr

Accessor to the related object on the forward side of a one-to-one relation.

In the example:

class Restaurant(Model):
    place = OneToOneField(Place, related_name='restaurant')

Restaurant.place is a ForwardOneToOneDescriptor instance.

digitalobject_ptr_id
job_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

resource_url

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

classmethod save_job_list(jobs: Iterable[Job], project: Project | None = None)

Store a list of Job s in the database as digital objects.

Optionally, if a Project is given, add the digital objects to this project.

unicore_resource_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class heliport.unicore.models.DBPathDir(*args, **kwargs)

Bases: UnicoreResource, DigitalObject, DirectoryObj

Django model for directory in unicore storage.

exception DoesNotExist

Bases: DoesNotExist

exception MultipleObjectsReturned

Bases: MultipleObjectsReturned

as_directory(context: Context) Directory

This makes it a directory in HELIPORT. Returns a UnicoreDirectory.

digitalobject_ptr

Accessor to the related object on the forward side of a one-to-one relation.

In the example:

class Restaurant(Model):
    place = OneToOneField(Place, related_name='restaurant')

Restaurant.place is a ForwardOneToOneDescriptor instance.

digitalobject_ptr_id
path

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

resource_url

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

unicore_resource_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class heliport.unicore.models.DBPathFile(*args, **kwargs)

Bases: UnicoreResource, DigitalObject, FileObj

Django model for file in unicore storage.

exception DoesNotExist

Bases: DoesNotExist

exception MultipleObjectsReturned

Bases: MultipleObjectsReturned

as_file(context: Context) File

This makes it a file in HELIPORT. Returns a UnicoreFile.

digitalobject_ptr

Accessor to the related object on the forward side of a one-to-one relation.

In the example:

class Restaurant(Model):
    place = OneToOneField(Place, related_name='restaurant')

Restaurant.place is a ForwardOneToOneDescriptor instance.

digitalobject_ptr_id
path

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

resource_url

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

unicore_resource_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class heliport.unicore.models.UnicoreClientFor(user: HeliportUser)

Bases: object

wraps unicore_client() in a heliport.core.utils.context.Description.

generate(context)
user: HeliportUser
class heliport.unicore.models.UnicoreDirectory(storage_url: str, path: str, context: Context)

Bases: UnicorePath, Directory

The actual directory with get_parts() implemented.

get_file_info() Dict[str, str]

Get last access date for the user to see.

get_parts() Iterable[FileOrDirectory]

Get the contents of this directory as UnicorePathDirObj or UnicorePathFileObj. Ignores hidden files and directories.

get_path_dir() PathDir

Pyunicore PathDir of this directory.

class heliport.unicore.models.UnicoreFile(storage_url: str, path: str, context: Context)

Bases: UnicorePath, File

An actual readable UNICORE file.

close()

Closes file if open; you can also use with block instead of calling this.

get_file_info() Dict[str, str]

Get the size and last accessed date for the user to see (if available).

get_path_file() PathFile

Get pyunicore PathFile of this object.

mimetype() str

Guess mimetype from file name.

open()

Opens the file for reading bytes; close() this afterwards.

read(number_of_bytes=None) bytes

Read bytes from this file; needs to be open().

size() int | None

File size in bytes.

class heliport.unicore.models.UnicorePath(storage_url: str, path: str, context: Context)

Bases: FileOrDirectory, ABC

Base class for paths inside a UNICORE storage.

context: Context

Context to cache the connection and stat infos

get_client() Client

Get or create a UNICORE client from the current context.

get_storage() Storage

The UNICORE storage in which this file can be found under path.

path: str

The path of this file in the storage returned by get_storage()

property path_obj

The pathlib path of path.

storage_url: str

The unicore api url for a storage that contains the path

class heliport.unicore.models.UnicorePathDirObj(storage_url: str, path: str, label: str | None = None)

Bases: UnicorePathObj, DirectoryObj

Represent a UNICORE directory that is not in the HELIPORT database.

as_digital_object(context: Context) DigitalObject

Return this as a DBPathDir in the database (create if none exists).

as_directory(context: Context) Directory

Makes this usable as a directory in HELIPORT; returns UnicoreDirectory.

static resolve_digital_object(storage_url: str, path: str, user: HeliportUser | None) DBPathDir | None

Try to get a DBPathDir version of this directory.

static type_id() str

Used to identify this type during resolution.

class heliport.unicore.models.UnicorePathFileObj(storage_url: str, path: str, label: str | None = None)

Bases: UnicorePathObj, FileObj

Represents a file in unicore storage that is not in the HELIPORT database.

as_digital_object(context: Context) DigitalObject

Return this as a DBPathFile in the database (create if none exists).

as_file(context: Context) File

Makes this an object representing a file; returns a UnicoreFile.

static resolve_digital_object(storage_url: str, path: str, user: HeliportUser | None) DBPathFile | None

Try to find a matching DBPAthFile instance.

static type_id() str

Used in resolution to identify this type.

class heliport.unicore.models.UnicorePathObj(storage_url: str, path: str, label: str | None = None)

Bases: GeneralDigitalObject, ABC

Abstract base class for files and directory in unicore storages not stored in DB.

as_html() str

Represent just as string in html.

as_rdf() RootedRDFGraph

Dummy implementation that represents this as string.

as_text() str

Represent by label or filename if no label provided. So normally label is unnecessary. But storages itself have no file name (path is “”).

as_tuple()

For __hash__ and __eq__ implementation.

property for_job

Determine if this file or directory is created for a specific UNICORE job.

get_identifying_params() Dict[str, str]

Parameters which can be used to resolve this instance.

classmethod resolve(params: Dict[str, str], context: Context) GeneralDigitalObject | None

Resolve an instance by its identifying params (get_identifying_params()).

Tries to return something in the database before returning something not in DB.

abstract static resolve_digital_object(storage_url: str, path: str, user: HeliportUser | None) Any | None

Implement this in subclass to handle resolution to object in database.

class heliport.unicore.models.UnicoreResource(*args, **kwargs)

Bases: Model

Base class for UNICORE Resources as Digital Objects.

UnicoreResource can not inherit from DigitalObject because DigitalObjects don’t support subclasses of subclasses yet.

class Meta

Bases: object

abstract = False
resource_url

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

unicore_resource_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

heliport.unicore.models.unicore_client(user: HeliportUser) Client

Create a pyunicore Client for the given user.

heliport.unicore.tests module

class heliport.unicore.tests.StorageTests(methodName='runTest')

Bases: WithProject

test_delete()
test_list()
class heliport.unicore.tests.TestCallbackView(*args, **kwargs)

Bases: TestCase

Tests for unicore.views.CallbackView.

property default_claims
path = '/unicore/notify/'
post_notification(claims=None, path=None, data=None)

POST a notification to the endpoint.

test_as_by_unicore()

Correct requests as UNICORE would do them should receive HTTP 204.

test_data_missing_href()

Requests sending data without “href” field should receive HTTP 400.

test_expired_token()

Requests using an expired token should receive HTTP 403.

test_malformed_data()

Requests sending malformed JSON data should receive HTTP 400.

test_missing_jwt_claims()

Requests with missing required JWT claims should receive HTTP 403.

test_missing_slash()

Posting to the correct path but with the trailing slash missing must work, too.

test_no_auth()

Requests without Authorization header should receive HTTP 403.

test_not_post()

All request methods except POST should receive HTTP 405.

CONNECT is not tested as it doesn’t seem to be supported by the test client.

test_wrong_auth_method()

Requests using the wrong authentication method should receive HTTP 403.

test_wrong_auth_token()

Requests using a token which can’t be decoded should receive HTTP 403.

test_wrong_content_type()

Requests with the wrong Content-Type should receive HTTP 415.

heliport.unicore.urls module

heliport.unicore.views module

class heliport.unicore.views.CallbackView(*args, **kwargs)

Bases: UnicoreServerAuthenticationRequiredMixin, View

Can be called by UNICORE to notify HELIPORT about a job’s status.

dispatch(request, *args, **kwargs)
post(request, *args, **kwargs)

Method called on POST request.

For more info see django.views.generic.base.View.dispatch().

This method is can be called by a UNICORE instance to give updates about a job’s status. If the job can be associated with a user and a project, it is imported into the HELIPORT database. The user that owns the job is identified by the DN stored in the data field of their “UNICORE” LoginInfo. The project is identified by the job tags. The job is only imported and added to the project if the identified user is a member of said project.

class heliport.unicore.views.JobActionView

Bases: UnicoreMixin, View

View that allows the user to perform some actions on a UNICORE job.

Parameters:

allowed_actions (List[str]) – The actions that can be performed on a job

get(request, job_id, action)

Method called on GET request.

For more info see django.views.generic.base.View.dispatch().

class heliport.unicore.views.JobDetailView

Bases: HeliportObjectMixin, UnicoreMixin, DetailView

get_context_data(**kwargs)
model

alias of DBJob

pk_url_kwarg = 'job'
template_name = 'unicore/job.html'
class heliport.unicore.views.JobsView

Bases: HeliportProjectMixin, UnicoreMixin, TemplateView

List of the UNICORE jobs that belong to a project.

Parameters:

unicore_job_tag_project (str) – Jobs with this tag are automatically associated with this project.

get_context_data(**kwargs)

Create method context.

For more info see django.views.generic.base.ContextMixin.get_context_data().

setup(request, *args, **kwargs)
template_name = 'unicore/jobs.html'
class heliport.unicore.views.StorageView

Bases: HeliportProjectMixin, UnicoreMixin, DetailView

shows a list of storages and allows creation and editing.

from_url(url, context)

Create pyunicore Storage from url, reusing existing client in context.

get_context_data(**kwargs)

Called by Django to get rendering context; get files and directories.

label_for(storage)

Return a string label to present the storage.

model

alias of Project

pk_url_kwarg = 'project'
post(request, *args, **kwargs)

Handle post request; add or delete a storage.

template_name = 'unicore/storages.html'
class heliport.unicore.views.TestJobView

Bases: UnicoreMixin, View

View that, on behalf of the user, sends a job to the cluster.

This is meant for testing purposes. The job runs on the login node and executes the following code:

echo Hello World

The job will be tagged so that it can be found in the appropriate HELIPORT project.

get(request)

Method called on GET request.

For more info see django.views.generic.base.View.dispatch().

Module contents