Add canEncode validator for use in callback urls and bearer tokens

[whpearson]

To validate that we can encode with latin-1.

This is needed to stop unicode errors we are seeing in notifications-api when attempting
to send to these callback urls

[karlchillmaid]

Suggested change

	CanEncode(message="Urls have to be capable of being encoded in latin-1"),
	CanEncode(message="URL cannot include Unicode characters"),

[quis]

I don‘t know how much we care about technical correctness here but URLs can contain Unicode characters. Unicode contains basically all characters, including boring ones like a or 1.

When people try to use emoji in text messages we don’t get into the weeds about character sets. Instead we tell them which characters are the problem:

notifications-admin/app/main/validators.py

Lines 166 to 171 in 06fe9c3

	raise ValidationError(
	"You cannot use {} in text messages. {} will not display properly on some phones.".format(
	formatted_list(non_sms_characters, conjunction="or", before_each="", after_each=""),
	("It" if len(non_sms_characters) == 1 else "These characters"),
	)
	)

[whpearson]

Unicode characters have to be percent encoded, so perhaps that should be the correct error message? Happy to highlight the characters that should be.

[quis]

Yeah, if the error message was something like

🤪 and ŵ must be percent-encoded in URLs

that would be better

[whpearson]

"You cannot use the following characters in URIs. These characters, ∆ or 📲, might be mis-encoded."

I kept close to the sms error. What do you think @quis @karlchillmaid ?

[karlchillmaid]

How about this?

You cannot use ∆ in a web address. You must use percent encoding if you want to include this character in a URL.

You cannot use ∆ or 📲 in a web address. You must use percent encoding if you want to include these characters in a URL.

[whpearson]

I went with calling it a web address in in both places (for ease of coding/consistency)

[whpearson]

Removed the repetition now. So it is consistent with @karlchillmaid 's advice (in slack)

[quis]

These tests are good, but they only check that the CanEncode validator works as expected. They don’t test that it’s being used on the bearer token field.

[whpearson]

Pushed up a version that checks whether CallbackForms have the CanEncode validators attached to the fields. I'd rather not test the functionality twice (to avoid duplication of effort if it changed).

I couldn't find examples of this kind of code (our forms tests are quite small), so pointers on how to do it properly appreciated.