Skip to content
Snippets Groups Projects
Commit ba7c2479 authored by Reiter, Christoph's avatar Reiter, Christoph :snake:
Browse files

docs: Move the documentation into the dev-guide 

We need an admin/developer guide anyway, and much of what is documented here
would otherwise be duplicated there. Instead move the core docs to the dev-guide
and just to it here instead.

All other bundles still keep their docs in their respective bundle repos.
parent 297a2171
No related branches found
No related tags found
No related merge requests found
Pipeline #116118 passed
Stage: test
Show whitespace changes
Inline Side-by-side
README.md
+ 12
− 1
View file @ ba7c2479
...@@ -2,4 +2,15 @@ ...@@ -2,4 +2,15 @@
[GitLab](https://gitlab.tugraz.at/dbp/relay/dbp-relay-core-bundle) | [Packagist](https://packagist.org/packages/dbp/relay-core-bundle) [GitLab](https://gitlab.tugraz.at/dbp/relay/dbp-relay-core-bundle) | [Packagist](https://packagist.org/packages/dbp/relay-core-bundle)
Docs: see <https://gitlab.tugraz.at/dbp/relay/dbp-relay-core-bundle/-/tree/main/docs> The core bundle is the central bundle that needs to be installed in every Relay API
gateway and also is a dependency of every other API bundle.
* It provides functionality that is commonly needed by API bundles (error handling,
logging, etc)
* It integrates the auth bundle with the Symfony security system
* It provides console commands that API bundles can subscribe to
* It configures all dependencies to our needs (api-platform, symfony, etc.)
* and more ...
For more information on how to configure and interface with the core bundle see
the [Developer Guide](https://dbp-demo.tugraz.at/dev-guide/relay/dev/)
docs/config.md deleted 100644 → 0
+ 0
− 23
View file @ 297a2171
# Bundle Configuration
Created via `./bin/console config:dump-reference DbpRelayCoreBundle | sed '/^$/d'`
```yaml
# Default configuration for "DbpRelayCoreBundle"
dbp_relay_core:
# Some string identifying the current build (commit hash)
build_info: ~ # Example: deadbeef
# Some URL identifying the current build (URL to the commit on some git web interface)
build_info_url: ~ # Example: 'https://gitlab.example.com/project/-/commit/deadbeef'
# Path to the logo (256x256) of the API frontend
logo_path: ~ # Example: 'bundles/dbprelaycore/logo.png'
# The title text of the API docs page
docs_title: 'Relay API Gateway'
# The description text of the API docs page (supports markdown)
docs_description: '*part of the [Digital Blueprint](https://gitlab.tugraz.at/dbp) project*'
messenger_transport_dsn: '' # Deprecated (Since dbp/relay-core-bundle 0.1.20: Use "queue_dsn" instead.)
# See https://symfony.com/doc/5.3/messenger.html#redis-transport
queue_dsn: '' # Example: 'redis://localhost:6379'
# https://symfony.com/doc/5.3/components/lock.html
lock_dsn: '' # Example: 'redis://redis:6379'
```
docs/cron.md deleted 100644 → 0
+ 0
− 42
View file @ 297a2171
# Cron Jobs
The API gateway provides one shared cron command which you should call every few
minutes:
```bash
./bin/console dbp:relay:core:cron
```
For example in crontab, every 5 minutes:
```bash
*/5 * * * * /srv/api/bin/console dbp:relay:core:cron
```
This cron job will regularly prune caches and dispatch a cron event which can be
handled by different bundles.
To get access to such an event you have to implement an event listener:
```yaml
Dbp\Relay\MyBundle\Cron\CleanupJob:
tags:
- { name: kernel.event_listener, event: dbp.relay.cron }
```
The listener gets called with a `CronEvent` object. By calling
`CronEvent::isDue()` and passing an ID for logging and a [cron
expression](https://en.wikipedia.org/wiki/Cron) you get told when it is time to
run:
```php
class CleanupJob
{
public function onDbpRelayCron(CronEvent $event)
{
if ($event->isDue('mybundle-cleanup', '0 * * * *')) {
// Do cleanup things here..
}
}
}
```
docs/errors.md deleted 100644 → 0
+ 0
− 57
View file @ 297a2171
# API Errors and Error Handling
By default Symfony and API Platform convert `HttpException` and all subclasses
to HTTP errors with a matching status code. See
https://api-platform.com/docs/core/errors for details.
Since API Platform by default hides any message details for >= 500 and < 600 in
production and doesn't allow injecting any extra information into the resulting
JSON-LD error response we provide a special HttpException subclass which
provides those features.
The following will pass the error message to the client even in case the status
code is 5xx. Note that you have to be careful to not include any secrets in the
error message since they would be exposed to the client.
```php
use Dbp\Relay\CoreBundle\Exception\ApiError;
throw new APIError(500, 'My custom message');
```
which results in:
```json
{
"@context": "/contexts/Error",
"@type": "hydra:Error",
"hydra:title": "An error occurred",
"hydra:description": "My custom message"
}
```
Further more you can include extra information like an error ID and some extra
information in form of an object:
```php
use Dbp\Relay\CoreBundle\Exception\ApiError;
throw new APIError::withDetails(500, 'My custom message', 'my-id', ['foo' => 42]);
```
which results in:
```json
{
"@context": "/contexts/Error",
"@type": "hydra:Error",
"hydra:title": "An error occurred",
"hydra:description": "My custom message",
"relay:errorId": "my-id",
"relay:errorDetails": {
"foo": 42
}
```
If you are using status codes <= 400 and are fine with just the message, then
using any of the builtin exception types is fine.
docs/index.md deleted 100644 → 0
+ 0
− 39
View file @ 297a2171
# About
The core bundle is the central bundle that needs to be installed in every API
gateway and also is a dependency of every other API bundle.
* It provides functionality that is commonly needed by API bundles (error handling,
logging, etc)
* It integrates the auth bundle with the Symfony security system
* It provides console commands that API bundles can subscribe to
* It configures all dependencies to our needs (api-platform, symfony, etc.)
* and more ...
# Overview
A minimal working relay API gateway consists of the core bundle and an auth bundle.
```mermaid
graph TD
style core_bundle fill:#606096,color:#fff
subgraph API Gateway
api(("API"))
auth_bundle("Auth Bundle")
core_bundle("Core Bundle")
core_bundle --> auth_bundle
api --> core_bundle
end
click auth_bundle "./#auth-bundle"
click user_session "./#usersession"
```
### Auth Bundle
The auth bundle takes care of user authentication and communicates with an OIDC
server, for example [Keycloak](https://www.keycloak.org). It creates the Symfony
user object and converts OAuth2 scopes to Symfony user roles used for
authorization.
docs/local_data.md deleted 100644 → 0
+ 0
− 92
View file @ 297a2171
#Local Data
Local data provides a mechanism to extend resource entities by attributes which are not part of the entities default set of attributes. Local data can be added in custom entity (post-)event subscribers.
##Adding Local Data Attributes to Existing Entities
Integraters have to make sure that local attributes requested by their client applications are added in the backend. This can be done in custom entity (post-)event subscribers:
```php
class EntityEventSubscriber implements EventSubscriberInterface
{
public static function getSubscribedEvents(): array
{
return [
EntityPostEvent::NAME => 'onPost',
];
}
public function onPost(EntityPostEvent $event)
{
$sourceData = $event->getSourceData();
$event->trySetLocalDataAttribute('foo', $sourceData->getFoo());
if ($event->isLocalDataAttributeRequested('bar')) {
$bar = $externalApi->getBar(); // expensive api call
$event->setLocalDataAttribute('bar', $bar);
}
}
}
```
Events of built-in entities provide a `getSourceData()` and a `getEntity()` method by convention, where
* `getSourceData()` provides the full set of available attributes for the entity
* `getEntity()` provides the entity itself
To set local data attributes:
* If you have the attribute value already at hand, call `trySetLocalDataAttribute` . It is safe because it sets the value only if the attribute was requested and not yet set by another event subscriber.
* If getting the attribute value is expensive, call `setLocalDataAttribute` only if `isLocalDataAttributeRequested` is `true`, i.e. if the attribute was actually requested and not yet set.
Note that local data values have to be serializable to JSON.
## Local Data requests
Local data can be requested using the `includeLocal` parameter provided by resource entity GET operations. The format is the following:
```php
includeLocal=<ResourceName>.<attributeName>,...
```
It is a comma-separated list of 0 ... n `<ResourceName>.<attributeName>` pairs, where `ResourceName` is the `shortName` defined in the `ApiResource` annotation of an entity. The list may contain attributes form different resources.
The backend will return an error if
* The `shortName` of the entity contains `.` or `,` characters
* The format of the `includeLocal` parameter value is invalid
* Any of the requested attributes was not provided by the backend
The backend will issue a warning if
* The backend tried to set an attribute which was not requested or tried to set a requested attribute multiple times (e.g. by different multiple event subscribers)
##Creating Local Data aware Entities
You can easily add local data to your Entity (`MyEntity`) by:
* Using the `LocalDataAwareTrait` in `MyEntity`
* Implementing the `LocalDataAwareInterface` in `MyEntity`
* Adding the `LocalData:output` group to the normalization context of `MyEntity`. For example:
```php
normalizationContext={"groups" = {"MyEntity:output", "LocalData:output"}}
```
* Adding an event dispatcher member variable of type `LocalDataAwareEventDispatcher` to your entity provider
* On GET-requests, passing the value of the `includeLocal` parameter to the event dispatcher
```php
$this->eventDispatcher->initRequestedLocalDataAttributes($includeParameter);
```
* Creating a (post-)event `MyEntityPostEvent` extending `LocalDataAwareEvent`, which you pass to the event dispatcher's `dispatch` method once your entity provider is done setting up a new instance of `MyEntity`:
```php
// get some data
$mySourceData = $externalApi->getSourceData($identifier);
// craete a new instance of MyEntity
$myEntity = new MyEntity();
// first, set the entity's default attributes:
$myEntity->setIdentifier($mySourceData->getIdentifier());
$myEntity->setName($mySourceData->getName());
// now, fire the event allowing event subscribers to add local data attributes
$postEvent = new MyEntityPostEvent($myEntity, $mySourceData);
$this->eventDispatcher->dispatch($postEvent, MyEntityPostEvent::NAME);
return $myEntity;
```
In case your entity has nested entities (sub-resources), your entity provider is responsible for passing the `includeLocal` parameter to sub-resource providers.
\ No newline at end of file
docs/locks.md deleted 100644 → 0
+ 0
− 49
View file @ 297a2171
# Locks
Locks are used to provide exclusive access to a shared resource.
The Relay API gateway optionally requires a locking backend. By default it uses
a local backend which is only suitable if all processes run on the same server.
Once your API runs on multiple servers you need to configure a remote/shared
locking backend.
## Configuration
In the bundle configuration set the `lock_dsn` key to a DSN supported by the
[Symfony Lock component](https://symfony.com/doc/current/components/lock.html)
At the moment we only support RedisStore and PdoStore:
### Redis
```yaml
# config/packages/dbp_relay_core.yaml
dbp_relay_core:
# redis[s]://[pass@][ip|host|socket[:port]]
lock_dsn: 'redis://localhost:6379'
```
### PDO
```yaml
# config/packages/dbp_relay_core.yaml
dbp_relay_core:
lock_dsn: 'mysql://user:secret@mariadb:3306/dbname'
```
This will create a `lock_keys` table in your database where the lock information
will be stored.
## Usage in Code
```php
// Retrieve a LockFactory instance via dependency injection
public function __construct(LockFactory $lockFactory) {
// Make sure to prefix the resource string to avoid conflicts with other bundles
$lock = $lockFactory->createLock(/* ... */);
// ...
}
```
For more information see
https://symfony.com/doc/current/components/lock.html#usage
\ No newline at end of file
docs/queue.md deleted 100644 → 0
+ 0
− 122
View file @ 297a2171
# Queued Tasks
The Relay API gateway optionally requires a queuing system, which means tasks
get queued in a central data store and worked on after a request has finished.
The tasks can be processes using one or more workers on multiple machines in
parallel.
This requires two extra deployment related tasks:
1) One or more worker tasks have to be run in the background and automatically
restarted if they stop.
2) On deployment the worker processes have to be restarted to use the new code.
## Configuration
In the bundle configuration set the `queue_dsn` key to a DSN supported by the
[Symfony messenger component](https://symfony.com/doc/current/messenger.html)
At the moment we only support the redis and doctrine transports:
### Redis
This transport requires the Redis PHP extension (>=4.3) and a running Redis server (^5.0).
```yaml
# config/packages/dbp_relay_core.yaml
dbp_relay_core:
# redis[s]://[pass@][ip|host|socket[:port]]
queue_dsn: 'redis://localhost:6379'
```
This creates a redis stream automatically when active.
### Doctrine
In case of doctrine you have to install `symfony/doctrine-messenger`
```bash
composer require symfony/doctrine-messenger
```
then create a doctrine connection and point the `queue_dsn` to that connection:
```yaml
# config/packages/doctrine.yaml
doctrine:
dbal:
connections:
my-queue-connection-name:
url: 'mysql://db:secret@mariadb:3306/db'
```
```yaml
# config/packages/dbp_relay_core.yaml
dbp_relay_core:
queue_dsn: 'doctrine://my-queue-connection-name'
```
I will automatically create a new database table when active.
## Run the workers
Start a worker using
```bash
./bin/console dbp:relay:core:queue:work my-worker-01
```
It will automatically exit after a specific amount 0f time or after a specific
number of processed tasks.
Note:
* You need to take care of restarting it automatically.
* Each active worker needs to have a unique name passed as the first argument
which should stay the same across restarts.
## Restart the workers
After deployment run
```bash
./bin/console dbp:relay:core:queue:restart
```
This will signal the workers to exit after the current task, which means they
will be restarted by supervisor and will run the newly deployed code.
## Manage Workers with Supervisor
Symfony
[recommends](https://symfony.com/doc/current/messenger.html#supervisor-configuration)
to use [Supervisor](http://supervisord.org/) to handle worker jobs and restart them.
```bash
sudo apt-get install supervisor
```
```ini
;/etc/supervisor/conf.d/queue-worker.conf
[program:queue-work]
command=php /path/to/your/app/bin/console dbp:relay:core:queue:work "%(program_name)s_%(process_num)02d"
user=user
numprocs=2
startsecs=0
autostart=true
autorestart=true
process_name=%(program_name)s_%(process_num)02d
```
Change `user` to the Unix user on your server.
## Testing your Setup
After everything is set up you can create a few dummy tasks and see if they get
handled by the workers:
```bash
./bin/console dbp:relay:core:queue:test --count 10 --delay 3
```
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment