Describe the task. It can be a feature, documentation, etc.
Upon initial experiments, we found the latents of the consecutive tokens have a cosine distance of 0.75.
Hence, we added a cosine loss to the consecutive forecasting latents.
Upon initial experiments, we show latents are more active and froct is not degrading.
Hedgedoc URL, if you are keeping notes, plots, logs in hedgedoc.
No response
Area
Describe the task. It can be a feature, documentation, etc.
Upon initial experiments, we found the latents of the consecutive tokens have a cosine distance of 0.75.
Hence, we added a cosine loss to the consecutive forecasting latents.
Upon initial experiments, we show latents are more active and froct is not degrading.
Hedgedoc URL, if you are keeping notes, plots, logs in hedgedoc.
No response
Area