CatalogWantedMirrorsLedgerOperating StatementDocsStudio statusCard system
alt.freeweights.nemotron / open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16k
From: laion <huggingface.co/laion>Newsgroups: alt.freeweights.nemotronSubject:

open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16k

Date: 2026-11-15Commit: 18d5352Signed: 2DC28CC7B26A283B ✓

Model with 1.7B parameters. Safetensors weights, 3.43 GB in all. Published by laion under Apache-2.0.

1.7B params3.43GBopen-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16ksafetensorspermissive
FactsParams1.7BSize3.43 GBFormatssafetensorsTasknot taggedLicenseApache-2.0From sourcepermissive
$ fw get laion/open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16k
# manifest sha256 86df68a1a29a… · 3.43 GB · 13 files
$ fw verify ./open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16k laion/open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16k
From sourceHosted (signed)laion/open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16kLast check: 2026-11-15 04:19 UTC
Run at homeGet help

Get help: custom fine-tune or private deploy. We review your setup before we quote.

Works with

Run8Fine-tune7Convert8Scan2

Runs with

Ranked by fit to this model (safetensors), then by GitHub stars.

TransformersLocalAItinygradTriton Inference ServerBentoMLCTranslate2ExLlamaV2LitServe
OverviewModel cardFilesUse itLicenseProvenanceModel tree

Overview

Size
3.43 GB
Weights
3.43 GB (safetensors)
Parameters
1.7B
Formats
safetensors
Task
not tagged
License
Apache-2.0 (permissive)
Files
13, each with a sha256
Publisher
laion
Added
2026-11-15
Section
alt.freeweights.nemotron
Lineage

Original release from the publisher.

Where these bytes come from
Publisher at the pinned commit→You

Sources are tried in order. A file counts only when its size and sha256 match the manifest. Sources are checked when you ask for them, not on a timer.

Sources
BadgeHostLast checkResult
From sourcehuggingface.co2026-11-15 04:19 UTCpassed

Model card

From the publisher at commit 18d5352. Shown as plain text with links; images open at the publisher.

Nemotron-CC-2024 HQ, 1.7B / 100B tokens, extended to 16k context#

16k long-context extension of laion/open-sci-ref-v0.02-1.7b-nemotron-hq-100B-4096. Final train loss 0.9778.

Context#

Part of a 2x2 study for the NeurIPS 2026 submission Strong Post-Training from Permissive, Reasoning-Dominant, Web-Scale Pretraining (rebuttal experiments), isolating web corpus from reasoning enrichment:

reasoning front-loadedno front-loading
MV webMV-100BTMV-noinstruct
Nemotron webnemotron-hq-mv_reasoning-100Bnemotron-hq-100B

The clean, unconfounded contrast is nemotron-hq-mv_reasoning-100B vs nemotron-hq-100B: same web substrate, same recipe, same tokenizer, same EOD convention — the only difference is the presence of the reasoning/instruct data.

Framing note. MixtureVitae's reasoning subset is itself permissively licensed, so the mv_reasoning model is a web-substrate swap (non-permissive web + permissive reasoning), not a fully non-permissive corpus. Please describe it as "Nemotron-CC web + MV reasoning subset".

Pretraining#

  • 1.7B dense, open-sci-ref protocol, GPT-NeoX tokenizer (vocab 50304, uint16)
  • 100B tokens, 23,842 iters, seq len 4096, RoPE theta 100k
  • WSD schedule, lr 4e-3, 20% linear cooldown, global batch 1024
  • 64 nodes x 4 GH200

Note on modeling_opensci.py#

Patched so OpensciModel.forward accepts **kwargs. Recent transformers passes num_items_in_batch down from Trainer for correct loss averaging, and OpensciForCausalLM.forward forwards **kwargs into both self.model(...) and self.loss_function(...). Without the patch training fails with TypeError: OpensciModel.forward() got an unexpected keyword argument 'num_items_in_batch'. Loss behaviour is unchanged.

Long-context extension (this model)#

YaRN scaling factor 4.0, 4096 -> 16384. One epoch of long_sft (SeaLong + LongAlign 30% / SmolTalk 70%), lr 2e-4 cosine, 2 nodes x 4 GH200. Identical recipe to the MV-100BT and MV-noinstruct 16k models, so all four cells of the 2x2 are comparable.

Files

13 files · 3.43 GBWeights 3.43 GB (safetensors). Sizes and sha256 from the signed manifest.
Hide all 13 files with their sha256
FileSizesha256
.gitattributes1.52 KB11ad7efa24975ee4b0c3c3a38ed18737f0658a5f75a0a96787b576a78a023361
README.md2.30 KB825fc4a4dd4d45d98fdb8167271eec7ddcd49af9226ec677ec9507ec3aa9a644
all_results.json203 Bf0e0c9095aa034e75635ae40e5229a7b18d52fc67c18b93fdf2534084ee5d166
config.json1.07 KB896a74ed44d116b04ec551f530402052c1fc64cfa2cda0f2da3bc310790c28a3
configuration_opensci.py11.2 KBd3d0e71d723af0bfb561cea4bcc30b6e72ad761d5828bc5910137c8fb05686e8
generation_config.json133 B168c611fb6cd9402c28d1d5f2829c2b118e356384c2271d3c44c65b31b2813f5
model.safetensors3.43 GBb5dfa358992206792db6eee8651a252b4cb467382414b2b32d2f194d9374cd9c
modeling_opensci.py46.0 KBcf603acb76bcbe41654dede4d3e933700c06e33480892410bb92c067df38b354
special_tokens_map.json473 B19052c2488feb55351187465884cfcfd846001c645340388a1fec30f35972c91
tokenizer.json3.56 MB8b10d313a0ce2c72973eb2a74c1d7b70f96a3d433d89201d28b8a712b18e12ab
tokenizer_config.json5.76 KBae404a408f3e15ff8253776f0cc79f89e0b8d3bbe78e94b774f4f9085da93589
train_results.json203 Bf0e0c9095aa034e75635ae40e5229a7b18d52fc67c18b93fdf2534084ee5d166
trainer_state.json57.5 KB04abefc0926177768dcf7ef0058770ea117a90a97d8474fb7818243e7616f150
VersionsEach time the pinned upstream commit or file list changed in this repo. Newest first.
Upstream commitDateContents
18d5352db0 pinned now2026-11-15 04:19 UTC13 files · 3.43 GB

Use it

$ fw get laion/open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16k
# manifest sha256 86df68a1a29a… · 3.43 GB · 13 files
$ fw verify ./open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16k laion/open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16k
fw.pyChecks the signature and every sha256, tries each source in order, resumes.
$ python3 fw.py get https://freeweights.allternit.com/manifests/laion/open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16k.json --dir ./open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16k $ python3 fw.py verify ./open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16k ./laion__open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16k.json
hfThe Hugging Face CLI with one setting. Same repo id.
$ HF_ENDPOINT=https://freeweights.allternit.com hf download laion/open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16k --revision 18d5352db0f22d93c90d62e1e130f173a5d53a51 --local-dir ./open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16k
Run at home

License

Name
Apache License 2.0
Id
Apache-2.0
Class
Permissive
Redistribution
yes
Listing
Hosted (signed)
Decided
License as declared by the publisher. No person read it. 2026-11-15
Text in
README.md
Text sha256
825fc4a4dd4d45d98fdb8167271eec7ddcd49af9226ec677ec9507ec3aa9a644
Terms gate
No extra terms beyond the license.

Read the license text

Licenses are checked against the license table. A custom license passes only after a person reads it.

Provenance

From: laion <huggingface.co/laion>Newsgroups: alt.freeweights.nemotronSubject: open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16kDate: 2026-11-15Commit: 18d5352Signed: 2DC28CC7B26A283B ✓
Upstream
laion/open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16k
Pinned commit
18d5352db0f22d93c90d62e1e130f173a5d53a51
Manifest
open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16k.json · .minisig
Manifest sha256
86df68a1a29a3a7749c3c2ad4201ca95b8395563266b88f3f9a8a75a5fb241e2
Signature
minisign, Ed25519, key 2DC28CC7B26A283B
Created
2026-11-15 04:19 UTC
Generator
freeweights-manifest/0.1.0
Takedowns
None.

How to verify

$ curl -O https://freeweights.allternit.com/manifests/laion/open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16k.json $ curl -O https://freeweights.allternit.com/manifests/laion/open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16k.json.minisig $ minisign -Vm open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16k.json -P RWQ7KGqyx4zCLXtUFOqvfv8L3g+8fpKJ1FsuxPyOYMF+4KufbUxKq1Oq $ shasum -a 256 <file> # compare with the Files tab

A manifest whose signature fails must be rejected. A file whose size or sha256 does not match must be discarded. Verify a download

Model tree

No related models in this catalog.Nothing here lists this model as its parent, and it names no parent.
Get it
Download
$ HF_ENDPOINT=https://freeweights.allternit.com hf download laion/open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16k
Open in Freeweights Studio

Pull, verify and run it in the Allternit app on Desktop, web or phone.

Signed manifest
minisign, key 2DC28CC7B26A283B. open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16k.json. Provenance. Takedown history: none.
Source
huggingface.co/laion/open-sci-ref-1.7b-nemotron-hq-100B-4096-longsft_16k

Related models

All of alt.freeweights.nemotron →
From: laion <huggingface.co/laion>Subject: a3-rl-laion_nemotron-gym-agent-calendar-80-8B
From: ServiceNow-AI <huggingface.co/ServiceNow-AI>Subject: Apriel-Nemotron-15b-Thinker
From: lmstudio-community <huggingface.co/lmstudio-community>Subject: Apriel-Nemotron-15b-Thinker-GGUF
From: nvidia <huggingface.co/nvidia>Subject: Nemotron-3-Diarization
From: nvidia <huggingface.co/nvidia>Subject: nemotron-3.5-asr-streaming-0.6b
From: bartowski <huggingface.co/bartowski>Subject: nvidia_OpenCodeReasoning-Nemotron-14B-GGUF