Add Google Ads Crawler to Search Engine Spiders

We need to exclude Google Ads Bots from our deny rules.

You can set up an allow rule for an IP group containing all IPs of googlebot.

<@1282641039298924606> Thank you for your help, but the Spider IP Group don't includes all Google Ads Bots. We've got the message from google that ads bots are blocked.

I'll just pop in for a moment:

We also rely heavily on the proper functions of google-ads bots.

If we create an ip-group and use as reference e.g: https://developers.google.com/static/search/apis/ipranges/special-crawlers.json

Does safeline parses the json automatically or do we have to parse it and add the content via script or even manually?

Only IP or CIDR format are allowed. SafeLine won't auto-parse it

Good to know.

I've added a test ip to the content-box and saved it afterwards. Unfortunately, the IP didnt save. (Screenshot 1)

Now I investigated the psql container.

I have two questions now:

• Data which is written via UI in the content-box, in which column should it have been written in psql? I assume ip_text, since ips expexcts JSON Additionally to mention: The IP added via UI was not stored in the corresponding db table either

• I updated the column ip_text with an example IP, but it won't show up in the UI
  (Screenshot 2)

Can you reproduce it?

Yes exactly.

UPDATE public.mgt_ip_group
SET ip_text = '192.168.1.100/32'
WHERE id = 3;

But the content still does not show up in the UI

<@1282641039298924606> could you add a feature request to extend the IP Groups with more automaticaly refreshing groups like search engine spider for example google special crawler list, bing bot list etc.?

Hi Carrie, thank you for your Answer i allready have done so before your answer but the target is an automaticaly update of the ip group.

it is an feature request for the future.

Best Regards
Georg