static-similarity-mrl-multilingual-v1

Static Embeddings with BERT Multilingual uncased tokenizer finetuned on various datasets#

This is a sentence-transformers model trained on the wikititles, tatoeba, talks, europarl, global_voices, muse, wikimatrix, opensubtitles, stackexchange, quora, wikianswers_duplicates, all_nli, simple_wiki, altlex, flickr30k_captions, coco_captions, nli_for_simcse and negation datasets. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, paraphrase mining, text classification, clustering, and more.

Read our Static Embeddings blogpost to learn more about this model and how it was trained.

• 0 Active Parameters: This model does not use any active parameters, instead consisting exclusively of averaging pre-computed token embeddings.
• 100x to 400x faster: On CPU, this model is 100x to 400x faster than common options like multilingual-e5-small. On GPU, it's 10x to 25x faster.
• Matryoshka: This model was trained with a Matryoshka loss, allowing you to truncate the embeddings for faster retrieval at minimal performance costs.
• Evaluations: See Evaluations for details on performance on NanoBEIR, embedding speed, and Matryoshka dimensionality truncation.
• Training Script: See train.py for the training script used to train this model from scratch.

See static-retrieval-mrl-en-v1 for an English static embedding model that has been finetuned specifically for retrieval tasks.

Model Details#

Model Description#

• Model Type: Sentence Transformer
• Maximum Sequence Length: inf tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity
• Training Datasets:
  • wikititles
  • tatoeba
  • talks
  • europarl
  • global_voices
  • muse
  • wikimatrix
  • opensubtitles
  • stackexchange
  • quora
  • wikianswers_duplicates
  • all_nli
  • simple_wiki
  • altlex
  • flickr30k_captions
  • coco_captions
  • nli_for_simcse
  • negation
• Languages: en, multilingual, ar, bg, ca, cs, da, de, el, es, et, fa, fi, fr, gl, gu, he, hi, hu, hy, id, it, ja, ka, ko, ku, lt, lv, mk, mn, mr, ms, my, nb, nl, pl, pt, ro, ru, sk, sl, sq, sr, sv, th, tr, uk, ur, vi, zh, hr
• License: apache-2.0

Model Sources#

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture#

SentenceTransformer(
  (0): StaticEmbedding(
    (embedding): EmbeddingBag(105879, 1024, mode='mean')
  )
)

Usage#

Direct Usage (Sentence Transformers)#

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("tomaarsen/static-similarity-mrl-multilingual-v1")
# Run inference
sentences = [
    'It is known for its dry red chili powder .',
    'It is popular for dry red chili powder .',
    'These monsters will move in large groups .',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

This model was trained with Matryoshka loss, allowing this model to be used with lower dimensionalities with minimal performance loss. Notably, a lower dimensionality allows for much faster downstream tasks, such as clustering or classification. You can specify a lower dimensionality with the truncate_dim argument when initializing the Sentence Transformer model:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/static-similarity-mrl-multilingual-v1", truncate_dim=256)
embeddings = model.encode([
    "I used to hate him.",
    "Раньше я ненавидел его."
])
print(embeddings.shape)
# => (2, 256)

Evaluation#

We've evaluated the model on 5 languages which have a lot of benchmarks across various tasks on MTEB.

We want to reiterate that this model is not intended for retrieval use cases. Instead, we evaluate on Semantic Textual Similarity (STS), Classification, and Pair Classification. We compare against the excellent and small multilingual-e5-small model.

Image: image

Across all measured languages, static-similarity-mrl-multilingual-v1 reaches an average 92.3% for STS, 95.52% for Pair Classification, and 86.52% for Classification relative to multilingual-e5-small.

Image: image

To make up for this performance reduction, static-similarity-mrl-multilingual-v1 is approximately ~125x faster on CPU and ~10x faster on GPU devices than multilingual-e5-small. Due to the super-linear nature of attention models, versus the linear nature of static embedding models, the speedup will only grow larger as the number of tokens to encode increases.

Matryoshka Evaluation#

Lastly, we experimented with the impacts on English STS on MTEB performance when we did Matryoshka-style dimensionality reduction by truncating the output embeddings to a lower dimensionality.

Image: English STS MTEB performance vs Matryoshka dimensionality reduction

As you can see, you can easily reduce the dimensionality by 2x or 4x with minor (0.15% or 0.56%) performance hits. If the speed of your downstream task or your storage costs are a bottleneck, this should allow you to alleviate some of those concerns.

Training Details#

Training Datasets#

wikititles

• Dataset: wikititles at d92a4d2
• Size: 14,700,458 training samples
• Columns: english and non_english
• Approximate statistics based on the first 1000 samples: | | english | non_english | |:--------|:----------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------| | type | string | string | | details | min: 4 characters

mean: 18.33 characters

max: 84 characters

| min: 4 characters

mean: 17.19 characters

max: 109 characters

|

• Samples: | english | non_english | |:------------------------|:---------------------------| | Le Vintrou | Ле-Вентру | | Greening | Begrünung | | Warrap | واراب (توضيح) |
• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          1024,
          512,
          256,
          128,
          64,
          32
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }

tatoeba

• Dataset: tatoeba at cec1343
• Size: 4,138,956 training samples
• Columns: english and non_english
• Approximate statistics based on the first 1000 samples: | | english | non_english | |:--------|:-----------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------| | type | string | string | | details | min: 5 characters

mean: 31.59 characters

max: 196 characters

| min: 6 characters

mean: 30.95 characters

max: 161 characters

|

• Samples: | english | non_english | |:-------------------------------------------------------|:-------------------------------------| | I used to hate him. | Раньше я ненавидел его. | | It is nothing less than an insult to her. | それはまさに彼女に対する侮辱だ。 | | I've apologized, so lay off, OK? | 謝ったんだから、さっきのはチャラにしてよ。 |
• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          1024,
          512,
          256,
          128,
          64,
          32
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }

talks

• Dataset: talks at 0c70bc6
• Size: 9,750,031 training samples
• Columns: english and non_english
• Approximate statistics based on the first 1000 samples: | | english | non_english | |:--------|:-----------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------| | type | string | string | | details | min: 5 characters

mean: 94.41 characters

max: 493 characters

| min: 4 characters

mean: 82.49 characters

max: 452 characters

|

• Samples: | english | non_english | |:------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------| | (Laughter) EC: But beatbox started here in New York. | (Skratt) EC: Fast beatbox började här i New York. | | I did not have enough money to buy food, and so to forget my hunger, I started singing." | 食べ物を買うお金もなかった だから 空腹を忘れるために 歌を歌い始めたの」 | | That is another 25 million barrels a day. | 那时还要增加两千五百万桶的原油。 |
• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          1024,
          512,
          256,
          128,
          64,
          32
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }

europarl

• Dataset: europarl at 11007ec
• Size: 4,990,000 training samples
• Columns: english and non_english
• Approximate statistics based on the first 1000 samples: | | english | non_english | |:--------|:------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------| | type | string | string | | details | min: 0 characters

mean: 147.77 characters

max: 668 characters

| min: 0 characters

mean: 153.13 characters

max: 971 characters

|

• Samples: | english | non_english | |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | (SK) I would like to stress three key points in relation to this issue. | (SK) Chtěla bych zdůraznit tři klíčové body, které jsou s tímto tématem spojeny. | | Women have a higher recorded rate of unemployment, especially long term unemployment. | Blandt kvinder registreres større arbejdsløshed, især blandt langtidsarbejdsløse. | | You will recall that we have occasionally had disagreements over how to interpret Rule 166 of our Rules of Procedure and that certain Members thought that the Presidency was not applying it properly, since it was not giving the floor for points of order that did not refer to the issue that was being debated at that moment. | De husker nok, at vi til tider har været uenige om fortolkningen af artikel 166 i vores forretningsorden, og at nogle af medlemmerne mente, at formanden ikke anvendte den korrekt, eftersom han ikke gav ordet til indlæg til forretningsordenen, når det ikke drejede sig om det spørgsmål, der blev drøftet på det pågældende tidspunkt. |
• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          1024,
          512,
          256,
          128,
          64,
          32
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }

global_voices

• Dataset: global_voices at 4cc20ad
• Size: 1,099,099 training samples
• Columns: english and non_english
• Approximate statistics based on the first 1000 samples: | | english | non_english | |:--------|:------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------| | type | string | string | | details | min: 5 characters

mean: 115.13 characters

max: 740 characters

| min: 3 characters

mean: 119.89 characters

max: 801 characters

|

• Samples: | english | non_english | |:------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------| | Generation 9/11: Cristina Balli (USA) from British Council USA on Vimeo. | Генерација 9/11: Кристина Бали (САД) од Британскиот совет САД на Вимео. | | Jamaica: Mapping the state of emergency · Global Voices | Jamaica: Mapeando el estado de emergencia | | It takes more than courage or bravery to do such a... http://fb.me/12T47y0Ml | Θέλει κάτι παραπάνω από κουράγιο ή ανδρεία για να κάνεις κάτι τέτοιο... http://fb.me/12T47y0Ml |
• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          1024,
          512,
          256,
          128,
          64,
          32
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }

muse

• Dataset: muse at 238c077
• Size: 1,368,274 training samples
• Columns: english and non_english
• Approximate statistics based on the first 1000 samples: | | english | non_english | |:--------|:---------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------| | type | string | string | | details | min: 3 characters

mean: 7.38 characters

max: 16 characters

| min: 1 characters

mean: 7.33 characters

max: 18 characters

|

• Samples: | english | non_english | |:---------------------|:--------------------| | metro | metrou | | suggest | 제안 | | nnw | nno |
• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          1024,
          512,
          256,
          128,
          64,
          32
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }

wikimatrix

• Dataset: wikimatrix at 74a4cb1
• Size: 9,688,498 training samples
• Columns: english and non_english
• Approximate statistics based on the first 1000 samples: | | english | non_english | |:--------|:-------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------| | type | string | string | | details | min: 16 characters

mean: 124.31 characters

max: 418 characters

| min: 11 characters

mean: 129.99 characters

max: 485 characters

|

• Samples: | english | non_english | |:-------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------| | 3) A set of wikis to support collaboration activities and disseminate information about good practices. | 3) Un conjunt de wikis per donar suport a les activitats de col·laboració i difusió d'informació sobre bones pràctiques. | | Daily cruiseferry services operate to Copenhagen and Frederikshavn in Denmark, and to Kiel in Germany. | Dịch vụ phà du lịch hàng ngày vận hành tới Copenhagen và Frederikshavn tại Đan Mạch, và tới Kiel tại Đức. | | In late April 1943, Philipp was ordered to report to Hitler's headquarters, where he stayed for most of the next four months. | Sent i april 1943 fick Philipp ordern att rapportera till Hitlers högkvarter, där han stannade i fyra månader. |
• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          1024,
          512,
          256,
          128,
          64,
          32
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }

opensubtitles

• Dataset: opensubtitles at d86a387
• Size: 4,990,000 training samples
• Columns: english and non_english
• Approximate statistics based on the first 1000 samples: | | english | non_english | |:--------|:-----------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------| | type | string | string | | details | min: 0 characters

mean: 34.43 characters

max: 220 characters

| min: 0 characters

mean: 26.99 characters

max: 118 characters

|

• Samples: | english | non_english | |:------------------------------------------------------------------------|:---------------------------------------------------------------| | Would you send a tomato juice, black coffee and a masseur? | هل لك أن ترسل لي عصير طماطم قهوة سوداء.. والمدلك! | | To hear the angels sing | لكى تسمع غناء الملائكه | | Brace yourself. | " تمالك نفسك " بريكر |
• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          1024,
          512,
          256,
          128,
          64,
          32
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }

stackexchange

• Dataset: stackexchange at 1c9657a
• Size: 250,519 training samples
• Columns: post1 and post2
• Approximate statistics based on the first 1000 samples: | | post1 | post2 | |:--------|:--------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------| | type | string | string | | details | min: 77 characters

mean: 669.56 characters

max: 3982 characters

| min: 81 characters

mean: 641.44 characters

max: 4053 characters

|

• Samples: | post1 | post2 | |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | New user question about passwords Just got a refurbished computer with Ubuntu as the OS. Have never even heard of the OS and now I'm trying to learn. When I boot the system, it starts up great. But, if I try to navigate around, it requires a password. Is there a trick to finding the initial password? Please advise. | How do I reset a lost administrative password? I'm working on a Ubuntu system, and my client has completely forgotten his administrative password. He doesn't even remember entering one; however it is there. I've tried the suggestions on the website, and I have been unsuccessful in deleting the password so that I can download applets required for running some files. Is there a solution? | | Reorder a list of string randomly but constant in a period of time I need to reorder a list in a random way but I want to have the same result on a short period of time ... So I have: var list = new String[] { "Angie", "David", "Emily", "James" } var shuffled = list.OrderBy(v => "4a78926c")).ToList(); But I always get the same order ... I could use Guid.NewGuid() but then I would have a different result in a short period of time. How can I do this? | Randomize a List What is the best way to randomize the order of a generic list in C#? I've got a finite set of 75 numbers in a list I would like to assign a random order to, in order to draw them for a lottery type application. | | Made a mistake on check need help to fix I wrote a check and put the amount in the pay to order spot. Can I just mark it out, put the name in the spot and finish writing the check? | How to correct a mistake made when writing a check? I think I know the answer to this, but I'm not sure, and it's a good question, so I'll ask: What is the accepted/proper way to correct a mistake made on a check? For instance, I imagine that in any given January, some people accidentally date a check in the previous year. Is there a way to correct such a mistake, or must a check be voided (and wasted)? Pointers to definitive information (U.S., Canada, and elsewhere) are helpful. |
• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          1024,
          512,
          256,
          128,
          64,
          32
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }

quora

• Dataset: quora at 451a485
• Size: 101,762 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples: | | anchor | positive | negative | |:--------|:------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------| | type | string | string | string | | details | min: 16 characters

mean: 53.47 characters

max: 249 characters

| min: 16 characters

mean: 52.63 characters

max: 237 characters

| min: 14 characters

mean: 54.67 characters

max: 292 characters

|

• Samples: | anchor | positive | negative | |:------------------------------------------------------|:-------------------------------------------------|:----------------------------------------------------| | What food should I try in Brazil? | Which foods should I try in Brazil? | What meat should one eat in Argentina? | | What is the best way to get a threesome? | How does one find a threesome? | How is the experience of a threesome? | | Whether I do CA or MBA? Which is better? | Which is better CA or MBA? | Which is better CA or IT? |
• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          1024,
          512,
          256,
          128,
          64,
          32
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }

wikianswers_duplicates

• Dataset: wikianswers_duplicates at 9af6367
• Size: 9,990,000 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples: | | anchor | positive | |:--------|:------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------| | type | string | string | | details | min: 14 characters

mean: 47.39 characters

max: 151 characters

| min: 15 characters

mean: 47.58 characters

max: 154 characters

|

• Samples: | anchor | positive | |:----------------------------------------------------------------------|:-------------------------------------------------------------------------| | Did Democritus belive matter was continess? | Why did democritus call the smallest pice of matter atomos? | | Tell you about the most ever done to satisfy a customer? | How do you satisfy your client or customer? | | How is a chemical element different from a compound? | How is a chemical element different to a compound? |
• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          1024,
          512,
          256,
          128,
          64,
          32
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }

all_nli

• Dataset: all_nli at d482672
• Size: 557,850 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples: | | anchor | positive | negative | |:--------|:------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------| | type | string | string | string | | details | min: 18 characters

mean: 34.88 characters

max: 193 characters

| min: 15 characters

mean: 46.49 characters

max: 181 characters

| min: 16 characters

mean: 50.47 characters

max: 204 characters

|

• Samples: | anchor | positive | negative | |:---------------------------------------------------------------------------|:-------------------------------------------------|:-----------------------------------------------------------| | A person on a horse jumps over a broken down airplane. | A person is outdoors, on a horse. | A person is at a diner, ordering an omelette. | | Children smiling and waving at camera | There are children present | The kids are frowning | | A boy is jumping on skateboard in the middle of a red bridge. | The boy does a skateboarding trick. | The boy skates down the sidewalk. |
• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          1024,
          512,
          256,
          128,
          64,
          32
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }

simple_wiki

• Dataset: simple_wiki at 60fd9b4
• Size: 102,225 training samples
• Columns: text and simplified
• Approximate statistics based on the first 1000 samples: | | text | simplified | |:--------|:------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------| | type | string | string | | details | min: 18 characters

mean: 149.3 characters

max: 573 characters

| min: 16 characters

mean: 123.58 characters

max: 576 characters

|

• Samples: | text | simplified | |:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | The next morning , it had a small CDO and well-defined bands , and the system , either a weak tropical storm or a strong tropical depression , likely reached its peak . | The next morning , it had a small amounts of convection near the center and well-defined bands , and the system , either a weak tropical storm or a strong tropical depression , likely reached its peak . | | The region of measurable parameter space that corresponds to a regime is very often loosely defined . Examples include the superfluid regime '' , the steady state regime '' or `` the femtosecond regime '' . | This is common if a regime is threatened by another regime . | | The Lamborghini Diablo is a high-performance mid-engined sports car that was built by Italian automaker Lamborghini between 1990 and 2001 . | The Lamborghini Diablo is a sport car that was built by Lamborghini from 1990 to 2001 . |
• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          1024,
          512,
          256,
          128,
          64,
          32
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }

altlex

• Dataset: altlex at 97eb209
• Size: 112,696 training samples
• Columns: text and simplified
• Approximate statistics based on the first 1000 samples: | | text | simplified | |:--------|:-------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------| | type | string | string | | details | min: 13 characters

mean: 131.03 characters

max: 492 characters

| min: 13 characters

mean: 112.41 characters

max: 492 characters

|

• Samples: | text | simplified | |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------| | Reinforcement and punishment are the core tools of operant conditioning . | Principles of operant conditioning : | | The Japanese Ministry of Health , Labour and Welfare defines `` hikikomori '' as people who refuse to leave their house and , thus , isolate themselves from society in their homes for a period exceeding six months . | The Japanese Ministry of Health , Labour and Welfare defines hikikomori as people who refuse to leave their house for over six months . | | It has six rows of black spines and has a pair of long , clubbed spines on the head . | It has a pair of long , clubbed spines on the head . |
• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          1024,
          512,
          256,
          128,
          64,
          32
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }

flickr30k_captions#

• Dataset: flickr30k_captions at 0ef0ce3
• Size: 158,881 training samples
• Columns: caption1 and caption2
• Approximate statistics based on the first 1000 samples: | | caption1 | caption2 | |:--------|:------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------| | type | string | string | | details | min: 20 characters

mean: 63.19 characters

max: 318 characters

| min: 13 characters

mean: 63.65 characters

max: 205 characters

|

• Samples: | caption1 | caption2 | |:--------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------| | Four women pose for a photograph with a man in a bright yellow suit. | A group of friends get their photo taken with a man in a green suit. | | A many dressed in army gear walks on the crash walking a brown dog. | A man with army fatigues is walking his dog. | | Four people are sitting around a kitchen counter while one is drinking from a glass. | A group of people sit around a breakfast bar. |
• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          1024,
          512,
          256,
          128,
          64,
          32
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }

coco_captions

• Dataset: coco_captions at bd26018
• Size: 414,010 training samples
• Columns: caption1 and caption2
• Approximate statistics based on the first 1000 samples: | | caption1 | caption2 | |:--------|:------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------| | type | string | string | | details | min: 30 characters

mean: 52.57 characters

max: 151 characters

| min: 29 characters

mean: 52.71 characters

max: 186 characters

|

• Samples: | caption1 | caption2 | |:-------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------| | THERE ARE FRIENDS ON THE BEACH POSING | A group of people standing together on the beach while holding a woman. | | a lovely white bathroom with white shower curtain. | A white toilet sitting in a bathroom next to a sink. |

The publisher's README is longer; it is cut here. Read the rest at the source.