How do I delete evidence or time logs on a Task?

To delete evidence attached to a Task, go to the Evidence view of the Task.
​
Set the Task's status to In Progress:

Then, click for the quick-access menu on the Evidence and select Delete: