Missing email notifications in spite of email subscription settings #2549
Comments
|
Since the delivery task is returning without an error, I'm having trouble coming up with any ideas here except a continuation of the previous thread where the digest is not being addressed to the intended users. You can check via the mail plugin with This should return a sequence of user IDs that will be sent to. If this is empty, that's your problem. If not, let me know and we can try and dig in further. In the future, we'll probably end up adding more debug logging to this path to improve observability here. There are a few less likely cases, which all should end up logging if they occur, such as these ones (among others):
|
|
I have a similar issue. I (the first/super user) never get any email notifications. But other project members are getting the emails. |
|
@sul4bh: This was addressed in #2528 (specifically this comment.) |
|
@tkaemming I've just tried running your code and here's what it returns: I think that |
|
By the way, was just wondering if this could be of any use. This is the Celery Flower screenshot for the deliver_digest task I mentioned. There seems to be no error there.
|
|
Do you get a non-empty result back running this snippet? |
|
Yes, I've got a set of email addresses. In this case there's only 1 entry in the returned set because there's only 1 user subscribing to the project. |
|
@david17123 did you confirm there were no debug log messages? i.e. with celery worker -l DEBUG |
|
I haven't tried running with DEBUG level logging before. I just did and there are tons of things printed out, which I tried to copy and paste the tasks around receiving the event: |
|
@david17123 its possible this is getting evicted in Redis -- do you know what maxmemory is set to? I'm assuming Redis is being used for everything (broker included). |
|
Yes, redis is used for everything including the broker. I've checked the redis configuration file and I don't see maxmemory is explicitly set anywhere. When I checked via redis-cli it says that maxmemory is set to 0, which I assume means no limit. Furthermore, the stats also says that evicted_keys is 0. Is this the stats that you're after? |
|
We added some additional debug logging with #2569 (on master as of dd51c16) which could provide some visibility into what's going on here if you're up for deploying a git revision. I don't think this is part of the digest building process, but somewhere between when the digest data is handed off to to be rendered into and email message and delivered to you — this would at least confirm that. The key line here would be something like this: As long as |
|
I get a similar issue where the test email works from the Admin page, as does: project = Project.objects.get(slug='django')
message = MessageBuilder('test')
message.add_users((1,), project)
message.send()But I get no notification emails (all are turned on). Should I raise a new ticket for this, or do we think they're related? |
|
@joedborg can you confirm:
|
|
@dcramer >>> settings.SENTRY_DIGESTS
'sentry.digests.backends.redis.RedisBackend'I checked the membership as of issue 2528, which is fine (returns a populated object). |
|
As a simple temporary solution (until we can figure out what's going on here), you can put this in |
|
Done that and reloaded Sentry, but still not getting emails unfortunately. |
|
The only thing I can pick out of the logs is ...I don't think it's related though After which, just cycle through |
|
This looks more like it: I've obviously replaced my email and user |
|
When digests are disabled it doesnt even hit any of that code (which is what the Dummy backend does). The toronado log message is something that would show up when rendering an email. It really does sound like it's getting into the email pipeline: Can you confirm that this is returning members? I can't possible see how this would break, especially since we're not having any issues in prod and most of the standard mail path has not changed. |
|
@joedborg that is unrelated -- we use try insert pattern everywhere in Sentry so you'll commonly see that in SQL logs. In general, SQL logs are useless. |
|
@dcramer Ah ok, makes sense. >>> mail.get_send_to(project)
[1] |
|
@joedborg I find it very unlikely email would be breaking here, so can you also please confirm that a notification should be generated. That is, does the event match a rule to which it would even try to send something? |
|
@dcramer What else needs checking in the settings? I think it's all set to send (I've tried removing the events to trigger the new event email). |
|
We believe this was fixed with #2587. From the merge commit (8a7c726):
This resulted in every issue being filtered from the digest since each issue appeared as it was not in the unresolved state. (We don't send notifications for issues that are resolved or muted for reasons that are hopefully obvious.) |
Don't use unreliable identity check in digest pipeline. In some unknown cases, it's possible that the result of `record.value.event.group.get_status()` is a long rather than an integer. This results in the check `0L is 0` which is `False` (and not the correct answer to the intended check.) This will likely fix GH-2549.
Paranoia abounds after GH-2549.
|
The fix mentioned above is now available on PyPI as Sentry 8.0.4. If you continue to have issues after upgrading, please let us know. |
|
Sorry I haven't been able to follow this up for quite a while. Just want to confirm that the problem I had is now fixed (i.e. I can now receive emails) after upgrading to 8.0.4. Thanks for your very responsive and helpful support! |
|
8.0.4 fixed the problem for me as well. Thank you guys. |
This is a continuation of issue 2528 where email notifications are not received. However, this time the user is already properly subscribed to the projects and yet still no email (or digest) is coming. To begin with, here's the situation:
http://[sentry.domain]/organizations/sentry/members/1/; screenshot attached)sentry.tasks.digests.deliver_digestis fired and handled by celery worker (screenshot attached)This is the notification rules being applied. It is the default one created by Sentry, and it should create notification when a new event is seen.
This is a snippet of celery flower right after the event is received by sentry. It is clear that a deliver_digest task is fired at that moment. Also, it is shown in the kwargs column that the event is a new event.
Just to confirm that the user is subscribed for notifications for the project, the correct checkboxes are ticked.
I wonder what information did
deliver_digestprocess when it was executed on my sentry instance but I'm not too sure how to go that deep into the code to figure it out. As previously, I'm not even sure if this is actually a bug or merely a config issue. Any help would be appreciated! ThanksThe text was updated successfully, but these errors were encountered: