Scraping is done in-app based on a script created from JSON directives.

There are two parts to the script.

1. The Primary JSON that defines the location and priority of each Source JSON.
2. The Source SON that tells weyd how to extract information from the source.

Both of these must be accessible to weyd using HTTPS (SSL) from the device. They can be stored on a local computer/server, but the SSL Certificate must be valid. To make sure that they are accessible, it's best to open the link in a browser on your computer and make sure you don't get any Security Warnings/Errors.

You are free to use a URL shortener to make it easier to input the Scraper URL into weyd.

JSON packages are only loaded when the user adds/changes the Scraper URL, manually clicks Sync in the app, or restores Settings that contain the URL. Therefore, a custom JSON package could be placed on a temporary web server long enough for weyd to cache the scripts.

If, when creating your scraper JSON, you need weyd to have additional features to pull the information from a source, feel free to open a ticket here or reach out to us on Discord or Telegram.