README.md
# Kirby 3/4 Robots.txt
[](https://travis-ci.com/bnomei/kirby3-robots-txt)
[](https://coveralls.io/github/bnomei/kirby3-robots-txt)
[](https://codeclimate.com/github/bnomei/kirby3-robots-txt)
[](https://twitter.com/bnomei)
Manage the [robots.txt](https://developers.google.com/search/reference/robots_txt) from the Kirby config file.
## Similar Robots.txt Plugins
- [kirby3-helpers](https://github.com/johannschopplich/kirby-helpers)
- [kirby3-wellknown](https://github.com/omz13/kirby3-wellknown)
## Related SEO Plugins
- [kirby3-seo](https://github.com/tobimori/kirby-seo)
- [kirby3-helpers](https://github.com/johannschopplich/kirby-helpers)
- [kirby-meta-knight](https://github.com/diesdasdigital/kirby-meta-knight)
## Commercial Usage
> <br>
> <b>Support open source!</b><br><br>
> This plugin is free but if you use it in a commercial project please consider to sponsor me or make a donation.<br>
> If my work helped you to make some cash it seems fair to me that I might get a little reward as well, right?<br><br>
> Be kind. Share a little. Thanks.<br><br>
> ‐ Bruno<br>
>
| M | O | N | E | Y |
|---|----|---|---|---|
| [Github sponsor](https://github.com/sponsors/bnomei) | [Patreon](https://patreon.com/bnomei) | [Buy Me a Coffee](https://buymeacoff.ee/bnomei) | [Paypal dontation](https://www.paypal.me/bnomei/15) | [Hire me](mailto:b@bnomei.com?subject=Kirby) |
## Installation
- unzip [master.zip](https://github.com/bnomei/kirby3-robots-txt/archive/master.zip) as folder `site/plugins/kirby3-robots-txt` or
- `git submodule add https://github.com/bnomei/kirby3-robots-txt.git site/plugins/kirby3-robots-txt` or
- `composer require bnomei/kirby3-robots-txt`
## Staging Server? Debug Mode = Disallow all
When you set the global Kirby `debug` config to `true` the plugin will disallow all indexing for all user-agents. This is especially useful on a staging server but you could consider xml-sitemap and rss-feed among other things as well.
> ⚠️⚠️⚠️ THIS MEANS IF YOU HAVE KIRBY'S DEBUG MODE ENABLED IN PRODUCTION ALL SEARCH ENGINES WILL BE BLOCKED FROM INDEXING YOUR SITE!
## Adding Sitemap Link to Robots.txt
This plugin will add the sitemap link **automatically** to the `robots.txt` file for most available SEO plugins. You can skip setting the `bnomei.robots-txt.sitemap` config value to `sitemap.xml` in that case.
## Setup
The plugin generates automatic defaults for the starterkit. You do not have to enter them in the config file. But if you would it would look like this.
**defaults for starterkit**
```php
<?php
return [
'bnomei.robots-txt.content' => null, // string or callback
'bnomei.robots-txt.sitemap' => null, // null (aka. AUTOMATIC) or string or callback
'bnomei.robots-txt.groups' => [ // array or callback
'*' => [ // user-agent
'disallow' => [
'/kirby/',
'/site/',
'/cdn-cgi/',
],
'allow' => [
'/media/',
]
]
]
];
```
**using a plain string**
```php
<?php
return [
'bnomei.robots-txt.content' => 'user-agent: *
disallow: /kirby/
disallow: /site/
disallow: /cdn-cgi/
allow: /media/',
];
```
**using a callback**
```php
<?php
return [
'bnomei.robots-txt.content' => function() {
return site()->myRobotsTxtContentField()->value();
},
];
```
**sitemap and multiple user-agents**
```php
<?php
return [
'bnomei.robots-txt.sitemap' => 'sitemap.xml',
'bnomei.robots-txt.groups' => [
'*' => [
'disallow' => [
'/',
],
],
'googlebot-images' => [
'allow' => [
'/media/',
]
]
]
];
```
## Disclaimer
This plugin is provided "as is" with no guarantee. Use it at your own risk and always test it yourself before using it in a production environment. If you find any issues, please [create a new issue](https://github.com/bnomei/kirby3-robots-txt/issues/new).
## License
[MIT](https://opensource.org/licenses/MIT)
It is discouraged to use this plugin in any project that promotes racism, sexism, homophobia, animal abuse, violence or any other form of hate speech.