Custom storage dependencies location

This guide describes how to create a custom storage adapter. However, it doesn’t mention where should dependencies be installed or come from. For example, I need aws-sdk module in my custom storage and also BaseAdapter should be used to extend the storage class. Per instructions, the storage is created in ghost-installation-root-dir/content/adapters/storage:

var BaseAdapter = require('ghost-storage-base');
const AWS = require('aws-sdk');

class Spaces extends BaseAdapter { ... }

Where is ghost-storage-base module supposed to come from?

Since there’s no package.json inside the root directory of ghost installation, if I install the module like yarn install aws-sdk, it will most likely be installed globally if it doesn’t find node_modules going up. That’s bad.

So how should dependencies be installed?

Most storages that I checked recommend this approach:

npm install ghost-storage-adapter-s3
mkdir -p ./content/adapters/storage
cp -r ./node_modules/ghost-storage-adapter-s3 ./content/adapters/storage/s3

where should the command npm i be executed? Judging by

cp -r ./node_modules/ghost-storage-adapter-s3 ./content/adapters/storage/s3

the assumption is that there’s node_modules folder in the root ghost directory, but there’s not package.json there, how can the node_modules directory be there?

I also found this

in ghost-google-cloud-storage, how is it possible that the node_modules be created without package.json?

If I were you I would make a new project folder for your custom storage adapter, and from that build a tarball which you can then install to your ghost directory.

This package.json might put you in the right direction:

{
  "name": "your-adapter-name",
  "version": "1.0.0",
  "scripts": {
    "build": "npm install && npm pack"
  },
  "dependencies": {
    "aws-sdk": "2.776.0",
    "ghost-storage-base": "0.0.5"
  },
  "bundledDependencies": [
    "aws-sdk",
    "ghost-storage-base"
  ]
}

You could use any way of building you like, but npm here is probably already installed on your system! The “bundledDependencies” array ensures that the node_modules will be bundled and copied with your custom adapter!

It might not be the canonical way to do it, but I hope this helps you!

thanks for the idea! could you elaborate on this

The “bundledDependencies” array ensures that the node_modules will be bundled and copied with your custom adapter!

also, I’m wondering about the canonical way to get the ghost-storage-base dependencies. Since it’s show in the documentation, somebody who wrote the doc probably had some workflow in mind.

Sure, so in your module you would install any dependencies, including ghost-storage-base using npm install

Then you would run npm build and it would generate a tar archive with your code AND the dependencies, in a local node_modules directory.

This archive could then be extracted to your content/adapters/storage directory and would have everything it needed to run. (see https://nodejs.org/docs/latest-v12.x/api/modules.html#modules_loading_from_node_modules_folders for how modules are loaded - TL;DR they use the closest node_modules directory that can be found, in this case it’ll be your bundled one)

I’m not sure how long ago that doc was written, maybe @Kevin would have a better idea about this?

1 Like

thanks, but having node_modules inside adapters/storage like this:

causes this error:

I assume that’s because ghost looks up sharp module in the adapters/storage/node_modules, whereas this package is installed in current/node_modules.

Any idea how to fix this?

Well, it’s sad that the docs don’t explain how to achieve this. It seems that the functionality to add custom storage adapters with dependencies through content/adapters is broken.

It worked for me to add all dependencies to current/package.json because that’s the directory ghost gets its dependencies from and put my own adapter to current/core/server/adapters/storage (the place where ghost keeps its own stuff). That’s of course is far from being an optimal solution, but at least it works.

@maxkoretskyi Could you share the entire error message? It’s hard to debug when we can only see part of it!

I assume that’s because ghost looks up sharp module in the adapters/storage/node_modules , whereas this package is installed in current/node_modules .

This shouldn’t be the case, Ghost has it’s own node_modules, and that is where Node would try to resolve sharp

I’m glad you’ve found a workaround for now, but I’d be interested to see exactly what was wrong with this :slight_smile:

1 Like

@fabien

Well, I just reinstalled Ghost locally to remove any remainders of my experimentation and tried the suggested approach and it worked. Thanks! I’m not sure what was messed up before, but probably some leftovers after experimenting with different approaches.

So is this the recommended approach? It’d be great to have a confirmation from the core team and maybe update the docs.

One thing I’m not sure about is whether there’s any advantaged to bundling dependencies. If there’s package.json in adapters/storage anyway, I could just run npm i inside the folder to get required modules. Maybe it makes sense if for some reason I don’t want to have package.json in adapters/storage or don’t want to install dependencies.

Ayyyy that’s awesome :dancer:

So is this the recommended approach? It’d be great to have a confirmation from the core team and maybe update the docs.

I’m going to raise this internally and try to get an official recommended approach put together, I understand the frustration!

One thing I’m not sure about is whether there’s any advantaged to bundling dependencies. If there’s package.json in adapters/storage anyway, I could just run npm i inside the folder to get required modules

That’s a great point, and totally valid! If you prefer to do that, or it makes more sense to you - then I say go for it :relaxed: The reason I thought to bundle them was to make it more of a simple install, particularly if you’re going to opensource it and allow others to use it.

On the other hand, anyone installing custom adapters is most likely technical enough to run an npm install command!

So you go with what you feel makes most sense, I’ll try to update this thread if/when I get anything about an official/recommended way!

1 Like

Great, thanks for your help!