From ba7c24794eaafd8b59bc8e9914d7ab8fdee9ecec Mon Sep 17 00:00:00 2001 From: Christoph Reiter <reiter.christoph@gmail.com> Date: Mon, 16 May 2022 16:17:52 +0200 Subject: [PATCH] docs: Move the documentation into the dev-guide We need an admin/developer guide anyway, and much of what is documented here would otherwise be duplicated there. Instead move the core docs to the dev-guide and just to it here instead. All other bundles still keep their docs in their respective bundle repos. --- README.md | 13 ++++- docs/config.md | 23 --------- docs/cron.md | 42 ---------------- docs/errors.md | 57 --------------------- docs/index.md | 39 --------------- docs/local_data.md | 92 ---------------------------------- docs/locks.md | 49 ------------------ docs/queue.md | 122 --------------------------------------------- 8 files changed, 12 insertions(+), 425 deletions(-) delete mode 100644 docs/config.md delete mode 100644 docs/cron.md delete mode 100644 docs/errors.md delete mode 100644 docs/index.md delete mode 100644 docs/local_data.md delete mode 100644 docs/locks.md delete mode 100644 docs/queue.md diff --git a/README.md b/README.md index 7549820..360bd99 100644 --- a/README.md +++ b/README.md @@ -2,4 +2,15 @@ [GitLab](https://gitlab.tugraz.at/dbp/relay/dbp-relay-core-bundle) | [Packagist](https://packagist.org/packages/dbp/relay-core-bundle) -Docs: see <https://gitlab.tugraz.at/dbp/relay/dbp-relay-core-bundle/-/tree/main/docs> +The core bundle is the central bundle that needs to be installed in every Relay API +gateway and also is a dependency of every other API bundle. + +* It provides functionality that is commonly needed by API bundles (error handling, + logging, etc) +* It integrates the auth bundle with the Symfony security system +* It provides console commands that API bundles can subscribe to +* It configures all dependencies to our needs (api-platform, symfony, etc.) +* and more ... + +For more information on how to configure and interface with the core bundle see +the [Developer Guide](https://dbp-demo.tugraz.at/dev-guide/relay/dev/) diff --git a/docs/config.md b/docs/config.md deleted file mode 100644 index cd3bc6d..0000000 --- a/docs/config.md +++ /dev/null @@ -1,23 +0,0 @@ -# Bundle Configuration - -Created via `./bin/console config:dump-reference DbpRelayCoreBundle | sed '/^$/d'` - -```yaml -# Default configuration for "DbpRelayCoreBundle" -dbp_relay_core: - # Some string identifying the current build (commit hash) - build_info: ~ # Example: deadbeef - # Some URL identifying the current build (URL to the commit on some git web interface) - build_info_url: ~ # Example: 'https://gitlab.example.com/project/-/commit/deadbeef' - # Path to the logo (256x256) of the API frontend - logo_path: ~ # Example: 'bundles/dbprelaycore/logo.png' - # The title text of the API docs page - docs_title: 'Relay API Gateway' - # The description text of the API docs page (supports markdown) - docs_description: '*part of the [Digital Blueprint](https://gitlab.tugraz.at/dbp) project*' - messenger_transport_dsn: '' # Deprecated (Since dbp/relay-core-bundle 0.1.20: Use "queue_dsn" instead.) - # See https://symfony.com/doc/5.3/messenger.html#redis-transport - queue_dsn: '' # Example: 'redis://localhost:6379' - # https://symfony.com/doc/5.3/components/lock.html - lock_dsn: '' # Example: 'redis://redis:6379' -``` diff --git a/docs/cron.md b/docs/cron.md deleted file mode 100644 index 543de07..0000000 --- a/docs/cron.md +++ /dev/null @@ -1,42 +0,0 @@ -# Cron Jobs - -The API gateway provides one shared cron command which you should call every few -minutes: - -```bash -./bin/console dbp:relay:core:cron -``` - -For example in crontab, every 5 minutes: - -```bash -*/5 * * * * /srv/api/bin/console dbp:relay:core:cron -``` - -This cron job will regularly prune caches and dispatch a cron event which can be -handled by different bundles. - -To get access to such an event you have to implement an event listener: - -```yaml - Dbp\Relay\MyBundle\Cron\CleanupJob: - tags: - - { name: kernel.event_listener, event: dbp.relay.cron } -``` - -The listener gets called with a `CronEvent` object. By calling -`CronEvent::isDue()` and passing an ID for logging and a [cron -expression](https://en.wikipedia.org/wiki/Cron) you get told when it is time to -run: - -```php -class CleanupJob -{ - public function onDbpRelayCron(CronEvent $event) - { - if ($event->isDue('mybundle-cleanup', '0 * * * *')) { - // Do cleanup things here.. - } - } -} -``` diff --git a/docs/errors.md b/docs/errors.md deleted file mode 100644 index 0fcf0b2..0000000 --- a/docs/errors.md +++ /dev/null @@ -1,57 +0,0 @@ -# API Errors and Error Handling - -By default Symfony and API Platform convert `HttpException` and all subclasses -to HTTP errors with a matching status code. See -https://api-platform.com/docs/core/errors for details. - -Since API Platform by default hides any message details for >= 500 and < 600 in -production and doesn't allow injecting any extra information into the resulting -JSON-LD error response we provide a special HttpException subclass which -provides those features. - -The following will pass the error message to the client even in case the status -code is 5xx. Note that you have to be careful to not include any secrets in the -error message since they would be exposed to the client. - -```php -use Dbp\Relay\CoreBundle\Exception\ApiError; - -throw new APIError(500, 'My custom message'); -``` - -which results in: - -```json -{ - "@context": "/contexts/Error", - "@type": "hydra:Error", - "hydra:title": "An error occurred", - "hydra:description": "My custom message" -} -``` - -Further more you can include extra information like an error ID and some extra -information in form of an object: - -```php -use Dbp\Relay\CoreBundle\Exception\ApiError; - -throw new APIError::withDetails(500, 'My custom message', 'my-id', ['foo' => 42]); -``` - -which results in: - -```json -{ - "@context": "/contexts/Error", - "@type": "hydra:Error", - "hydra:title": "An error occurred", - "hydra:description": "My custom message", - "relay:errorId": "my-id", - "relay:errorDetails": { - "foo": 42 - } -``` - -If you are using status codes <= 400 and are fine with just the message, then -using any of the builtin exception types is fine. diff --git a/docs/index.md b/docs/index.md deleted file mode 100644 index 9b3dad3..0000000 --- a/docs/index.md +++ /dev/null @@ -1,39 +0,0 @@ -# About - -The core bundle is the central bundle that needs to be installed in every API -gateway and also is a dependency of every other API bundle. - -* It provides functionality that is commonly needed by API bundles (error handling, - logging, etc) -* It integrates the auth bundle with the Symfony security system -* It provides console commands that API bundles can subscribe to -* It configures all dependencies to our needs (api-platform, symfony, etc.) -* and more ... - - -# Overview - -A minimal working relay API gateway consists of the core bundle and an auth bundle. - -```mermaid -graph TD - style core_bundle fill:#606096,color:#fff - - subgraph API Gateway - api(("API")) - auth_bundle("Auth Bundle") - core_bundle("Core Bundle") - core_bundle --> auth_bundle - api --> core_bundle - end - - click auth_bundle "./#auth-bundle" - click user_session "./#usersession" -``` - -### Auth Bundle - -The auth bundle takes care of user authentication and communicates with an OIDC -server, for example [Keycloak](https://www.keycloak.org). It creates the Symfony -user object and converts OAuth2 scopes to Symfony user roles used for -authorization. diff --git a/docs/local_data.md b/docs/local_data.md deleted file mode 100644 index 98366e8..0000000 --- a/docs/local_data.md +++ /dev/null @@ -1,92 +0,0 @@ -#Local Data - -Local data provides a mechanism to extend resource entities by attributes which are not part of the entities default set of attributes. Local data can be added in custom entity (post-)event subscribers. - -##Adding Local Data Attributes to Existing Entities - -Integraters have to make sure that local attributes requested by their client applications are added in the backend. This can be done in custom entity (post-)event subscribers: - -```php -class EntityEventSubscriber implements EventSubscriberInterface -{ - public static function getSubscribedEvents(): array - { - return [ - EntityPostEvent::NAME => 'onPost', - ]; - } - - public function onPost(EntityPostEvent $event) - { - $sourceData = $event->getSourceData(); - $event->trySetLocalDataAttribute('foo', $sourceData->getFoo()); - - if ($event->isLocalDataAttributeRequested('bar')) { - $bar = $externalApi->getBar(); // expensive api call - $event->setLocalDataAttribute('bar', $bar); - } - } -} -``` -Events of built-in entities provide a `getSourceData()` and a `getEntity()` method by convention, where -* `getSourceData()` provides the full set of available attributes for the entity -* `getEntity()` provides the entity itself - -To set local data attributes: -* If you have the attribute value already at hand, call `trySetLocalDataAttribute` . It is safe because it sets the value only if the attribute was requested and not yet set by another event subscriber. -* If getting the attribute value is expensive, call `setLocalDataAttribute` only if `isLocalDataAttributeRequested` is `true`, i.e. if the attribute was actually requested and not yet set. - -Note that local data values have to be serializable to JSON. - -## Local Data requests - -Local data can be requested using the `includeLocal` parameter provided by resource entity GET operations. The format is the following: - -```php -includeLocal=<ResourceName>.<attributeName>,... -``` - -It is a comma-separated list of 0 ... n `<ResourceName>.<attributeName>` pairs, where `ResourceName` is the `shortName` defined in the `ApiResource` annotation of an entity. The list may contain attributes form different resources. - -The backend will return an error if -* The `shortName` of the entity contains `.` or `,` characters -* The format of the `includeLocal` parameter value is invalid -* Any of the requested attributes was not provided by the backend - -The backend will issue a warning if -* The backend tried to set an attribute which was not requested or tried to set a requested attribute multiple times (e.g. by different multiple event subscribers) - -##Creating Local Data aware Entities - -You can easily add local data to your Entity (`MyEntity`) by: - -* Using the `LocalDataAwareTrait` in `MyEntity` -* Implementing the `LocalDataAwareInterface` in `MyEntity` -* Adding the `LocalData:output` group to the normalization context of `MyEntity`. For example: - ```php - normalizationContext={"groups" = {"MyEntity:output", "LocalData:output"}} - ``` -* Adding an event dispatcher member variable of type `LocalDataAwareEventDispatcher` to your entity provider -* On GET-requests, passing the value of the `includeLocal` parameter to the event dispatcher -```php -$this->eventDispatcher->initRequestedLocalDataAttributes($includeParameter); -``` -* Creating a (post-)event `MyEntityPostEvent` extending `LocalDataAwareEvent`, which you pass to the event dispatcher's `dispatch` method once your entity provider is done setting up a new instance of `MyEntity`: -```php -// get some data -$mySourceData = $externalApi->getSourceData($identifier); - -// craete a new instance of MyEntity -$myEntity = new MyEntity(); -// first, set the entity's default attributes: -$myEntity->setIdentifier($mySourceData->getIdentifier()); -$myEntity->setName($mySourceData->getName()); - -// now, fire the event allowing event subscribers to add local data attributes -$postEvent = new MyEntityPostEvent($myEntity, $mySourceData); -$this->eventDispatcher->dispatch($postEvent, MyEntityPostEvent::NAME); - -return $myEntity; -``` - -In case your entity has nested entities (sub-resources), your entity provider is responsible for passing the `includeLocal` parameter to sub-resource providers. \ No newline at end of file diff --git a/docs/locks.md b/docs/locks.md deleted file mode 100644 index edae5c8..0000000 --- a/docs/locks.md +++ /dev/null @@ -1,49 +0,0 @@ -# Locks - -Locks are used to provide exclusive access to a shared resource. - -The Relay API gateway optionally requires a locking backend. By default it uses -a local backend which is only suitable if all processes run on the same server. -Once your API runs on multiple servers you need to configure a remote/shared -locking backend. - -## Configuration - -In the bundle configuration set the `lock_dsn` key to a DSN supported by the -[Symfony Lock component](https://symfony.com/doc/current/components/lock.html) - -At the moment we only support RedisStore and PdoStore: - -### Redis - -```yaml -# config/packages/dbp_relay_core.yaml -dbp_relay_core: - # redis[s]://[pass@][ip|host|socket[:port]] - lock_dsn: 'redis://localhost:6379' -``` - -### PDO - -```yaml -# config/packages/dbp_relay_core.yaml -dbp_relay_core: - lock_dsn: 'mysql://user:secret@mariadb:3306/dbname' -``` - -This will create a `lock_keys` table in your database where the lock information -will be stored. - -## Usage in Code - -```php -// Retrieve a LockFactory instance via dependency injection -public function __construct(LockFactory $lockFactory) { - // Make sure to prefix the resource string to avoid conflicts with other bundles - $lock = $lockFactory->createLock(/* ... */); - // ... -} -``` - -For more information see -https://symfony.com/doc/current/components/lock.html#usage \ No newline at end of file diff --git a/docs/queue.md b/docs/queue.md deleted file mode 100644 index 4bf24c3..0000000 --- a/docs/queue.md +++ /dev/null @@ -1,122 +0,0 @@ -# Queued Tasks - -The Relay API gateway optionally requires a queuing system, which means tasks -get queued in a central data store and worked on after a request has finished. -The tasks can be processes using one or more workers on multiple machines in -parallel. - -This requires two extra deployment related tasks: - -1) One or more worker tasks have to be run in the background and automatically - restarted if they stop. -2) On deployment the worker processes have to be restarted to use the new code. - -## Configuration - -In the bundle configuration set the `queue_dsn` key to a DSN supported by the -[Symfony messenger component](https://symfony.com/doc/current/messenger.html) - -At the moment we only support the redis and doctrine transports: - -### Redis - -This transport requires the Redis PHP extension (>=4.3) and a running Redis server (^5.0). - -```yaml -# config/packages/dbp_relay_core.yaml -dbp_relay_core: - # redis[s]://[pass@][ip|host|socket[:port]] - queue_dsn: 'redis://localhost:6379' -``` - -This creates a redis stream automatically when active. - -### Doctrine - -In case of doctrine you have to install `symfony/doctrine-messenger` - -```bash -composer require symfony/doctrine-messenger -``` - -then create a doctrine connection and point the `queue_dsn` to that connection: - -```yaml -# config/packages/doctrine.yaml -doctrine: - dbal: - connections: - my-queue-connection-name: - url: 'mysql://db:secret@mariadb:3306/db' -``` - -```yaml -# config/packages/dbp_relay_core.yaml -dbp_relay_core: - queue_dsn: 'doctrine://my-queue-connection-name' -``` - -I will automatically create a new database table when active. - -## Run the workers - -Start a worker using - -```bash -./bin/console dbp:relay:core:queue:work my-worker-01 -``` - -It will automatically exit after a specific amount 0f time or after a specific -number of processed tasks. - -Note: - -* You need to take care of restarting it automatically. -* Each active worker needs to have a unique name passed as the first argument - which should stay the same across restarts. - - -## Restart the workers - -After deployment run - -```bash -./bin/console dbp:relay:core:queue:restart -``` - -This will signal the workers to exit after the current task, which means they -will be restarted by supervisor and will run the newly deployed code. - - -## Manage Workers with Supervisor - -Symfony -[recommends](https://symfony.com/doc/current/messenger.html#supervisor-configuration) -to use [Supervisor](http://supervisord.org/) to handle worker jobs and restart them. - -```bash -sudo apt-get install supervisor -``` - -```ini -;/etc/supervisor/conf.d/queue-worker.conf -[program:queue-work] -command=php /path/to/your/app/bin/console dbp:relay:core:queue:work "%(program_name)s_%(process_num)02d" -user=user -numprocs=2 -startsecs=0 -autostart=true -autorestart=true -process_name=%(program_name)s_%(process_num)02d -``` - -Change `user` to the Unix user on your server. - -## Testing your Setup - -After everything is set up you can create a few dummy tasks and see if they get -handled by the workers: - -```bash -./bin/console dbp:relay:core:queue:test --count 10 --delay 3 -``` -- GitLab