dev/joomla#54 New menu-based URL for HTTP method of calling cron

[webmaster-cses-org-uk]

Overview

As part of work on Joomla 5 compatibility (dev/joomla#54), one key problem identified is that using cli.php or cron.php boots CiviCRM first, before the CMS (Joomla). This is the opposite way round to when the web app (site) is accessed. Consequently, the order of precedence when autoloading classes and dependencies is wrong / different, and this causes the cron job to fail on J5+.

This PR is a proposed method to fix the problem for cron.php, by providing a route to design it out.

Note that #33825 provides a fix for cli.php.

Before

If you wish to execute your cron jobs via the HTTP method, you call a URL as follows:

https://example.org/administrator/components/com_civicrm/civicrm/bin/cron.php

This is an external script that has to manually bootstrap CiviCRM and then the CMS. It does not currently work on J4+.

After

Inspired by what has been done for the PayPal IPN notify URL, you can now call the following URL to execute cron jobs:

https://example.org/?option=com_civicrm&task=civicrm/cron/runjobs

This is now just a normal CiviCRM menu action / page access that therefore follows the usual boot sequence.

Longer-term, this brings the HTTP cron action into line with the rest of CiviCRM and allows us to design out cron.php if so desired.

Technical Details

Parameters passed to the call are the same (name, pass, key etc). Only the URL changes.

Tested and working on CiviCRM 6.6.1 (+ various J5 compatibility mods covered by other PRs) and Joomla 5.4.1.

The proposed solution adds a new method, CRM_Utils_System::executeScheduledJobsAsCron(), which largely copies the code from cron.php, updated as necessary.

For absolute clarity, it is also proposed to rename the existing method executeScheduledJobs() to executeScheduledJobsAsAdmin() (and update menu XML accordingly).

Comments

Although this PR presents a solution for Joomla, in principle this would work on any CMS (although may require other changes - not sure how well cron.php is supported on other platforms).

This PR is presented as a proof-of-concept. If it is agreed to adopt, the documentation will need updating (I can do this in due course).

Possibly an alternative solution is to use the REST API interface? (Link in documentation broken, think it should go to this page: https://docs.civicrm.org/dev/en/latest/api/v4/rest/)

[civibot]

🤖 Thank you for contributing to CiviCRM! ❤️ We will need to test and review this PR. 👷

Introduction for new contributors...

• If this is your first PR, an admin will greenlight automated testing with the command ok to test or add to whitelist.
• A series of tests will automatically run. You can see the results at the bottom of this page (if there are any problems, it will include a link to see what went wrong).
• A demo site will be built where anyone can try out a version of CiviCRM that includes your changes.
• If this process needs to be repeated, an admin will issue the command test this please to rerun tests and build a new demo site.
• Before this PR can be merged, it needs to be reviewed. Please keep in mind that reviewers are volunteers, and their response time can vary from a few hours to a few weeks depending on their availability and their knowledge of this particular part of CiviCRM.
• A great way to speed up this process is to "trade reviews" with someone - find an open PR that you feel able to review, and leave a comment like "I'm reviewing this now, could you please review mine?" (include a link to yours). You don't have to wait for a response to get started (and you don't have to stop at one!) the more you review, the faster this process goes for everyone 😄
• To ensure that you are credited properly in the final release notes, please add yourself to contributor-key.yml
• For more information about contributing, see CONTRIBUTING.md.

Quick links for reviewers...

• ➡️ PR Demo site (can be rebuilt with the command test this please).
• 📖 Review standards
• 🗒️ Review template (brief or verbose)

➡️ Online demo of this PR 🔗

[civibot]

The issue associated with the Pull Request can be viewed at https://lab.civicrm.org/dev/joomla/-/issues/54

[seamuslee001]

@webmaster-cses-org-uk I like your thinking here but I'm also wondering could you just use the civicrm/api3 route with url parameters of job.execute and appropriate keys that way?

[totten]

Parameters passed to the call are the same (name, pass, key etc). Only the URL changes.

I think it's interesting to consider whether these parameters are part of the ideal setup process. In my mind, I think the "System Status" would show you an alert and lead to some concrete suggestion for setup, e.g. (pseudo-text)

Cron Setup

Your CiviCRM site is not running cron jobs. Please follow one of the suggestions to enable cron:

• HTTP Cron: For a simple solution that is compatible with most servers, copy this to /etc/crontab:

  */15 * * * * root curl https://example.com/...civicrm/cron?secret=ABCD1234
  

  The /etc/crontab can be configured anywhere. It does NOT need to run directly on your web-server. However, this solution may not perform well if you have large jobs. It will be subject to the same resource-limits as your other web-requests.

• Local CLI Cron: For a more performant solution, copy this to /etc/crontab:

  */15 * * * * www-data (cd /var/www/example.com/web && cv cron --url=https://example.com)
  

  This solution works with small and large jobs. It is allowed to use more resources than HTTP. However, it must be configured directly on the web-server.

This kind of notification is an improved experience because the admin doesn't need to guess at details or cross-reference docs or manually configure users/passwords/roles/permissions.

Of course, you do need some identity for cron. (There will be logs, etc). For the recent implementation of cv cron, we used this. (See also: cv#246, cv#254, and cv#258.)

And for HTTP, you do need some protection (so that random bots don't create a DoS through your cron endpoint). It's ideal to have a single-purpose key.

I suspect that something along these lines would work:

1. Pre-generate key (e.g. could be installer or upgrader; or part of a status-check)

   if (empty(Civi::settings()->get('cron_key')) {
    Civi::settings()->set('cron_key', CRM_Utils_String::createRandom(24, CRM_Utils_String::ALPHANUMERIC))
   }

2. In the page-controller, don't call CRM_Utils_System::authenticateScript(TRUE, NULL, NULL, TRUE, FALSE, TRUE);. Instead:

   if (!hash_equals($_REQUEST['secret'], Civi::settings()->get('cron_key'))) {
     fail("Incorrect secret");
   }
   
   \CRM_Core_Config::singleton()->userPermissionTemp = new class extends \CRM_Core_Permission_Temp {
   
     public function check($perm) {
       return TRUE;
     }
   
   };
   
   $cid = NULL;
   if ($domain = \CRM_Core_BAO_Domain::getDomain()) {
     $cid = $domain->contact_id;
   }
   if (!$cid) {
     $cid = \CRM_Core_DAO::singleValueQuery('SELECT contact_id FROM civicrm_domain ORDER BY id LIMIT 1');
   }
   authx_login(['principal' => ['contactId' => $cid]]);

[totten]

Aside: it may not have come through quite right, but I should -- I think adding a route for the cron handler is a really good idea. 😄