rl-transformers

Contrastive BERT

Overview of CoBERL CoBERL, or Contrastive BERT, is a reinforcement learning agent that aims to improve data efficiency for RL. It achieves this by using a new contrastive loss and a hybrid LSTM-Transformer architecture. RL, or reinforcement learning, is a type of machine learning that involves an agent learning to make decisions by receiving feedback in the form of rewards or punishments. However, RL can be inefficient when it comes to using data, which is where CoBERL comes in. The Architec

Gated Transformer-XL

Introduction to GTrXL GTrXL is a new architecture for reinforcement learning based on the popular transformer model. This architecture introduces a few key architectural modifications to improve the stability and learning speed of the original transformer and XL variant. Key Modifications of GTrXL A few key modifications are introduced in GTrXL to improve its performance. One of the modifications is the placement of layer normalization on only the input stream of the submodules. This change

1 / 1