zbj"0SrSSKJrJrJr SSKrSSKJrJ r J r "SS\ R5r "SS\ R5r "S S \ R5r"S S \ R5r"S S\ R5r"SS\ R5r"SS\ R$5r"SS\ R(5r"SS\ R,5r"SS\ R05r"SS\ R,5r"SS\ R05rg)z0Declares specification of the Transformer model.)OptionalTupleUnionN)attention_spec common_spec model_specc/X\rSrSrSS\R R S\RRSSSSSSSSSSSSSSSS4S\ S\ S \ S \ S \R S \ S \RS\ S\ S\ S\ S\ S\ S\ \ S\ \ S\ \ S\ S\ \ RS\S\S\ \ S\ \ S\ 4.SjjrSrg) TransformerEncoderSpec TFN' num_layers num_headspre_norm no_final_norm activationnum_source_embeddingsembeddings_mergelayernorm_embeddingrelative_positionrelative_attention_biasffn_glurms_normmulti_query_attention num_heads_kvhead_dim rotary_dimrotary_interleaverotary_scaling_typerotary_scaling_factor rotary_basesliding_windowqk_normpre_post_layer_normchU (aUbUS:wa [S5eSnXl[R"S5R U5UlX0l[R"S5R U5Ul[R"S5R U5Ul[U5Vs/sHn[R"5PM snUl SUl U (dU (d[5UlU(a U(d[R "U S9UlU(a[R "U S9UlUb*[R"S5R U5Ul[U5Vs/sHn[)U U U U UUUUUUUUUUS 9PM snUlgs snfs snf) a/Initializes a Transformer encoder specification. Args: num_layers: Number of layers. num_heads: Number of attention heads. pre_norm: Enable the pre-norm Transformer architecture. no_final_norm: Disable the final layer norm in the pre-norm architecture. activation: Activation to apply in the feed-forward network. num_source_embeddings: Number of source embeddings. embeddings_merge: When :obj:`num_source_embeddings` > 1, specify how the embeddings are merged. layernorm_embedding: Apply layer normalization after the embedding layer. relative_position: Use relative position representations in the self-attention layers as described in https://arxiv.org/abs/1803.02155. relative_attention_bias: Use relative attention bias in the self-attention layers as described in the T5 paper https://arxiv.org/abs/1910.10683. ffn_glu: Use gated linear units in the FFN layers as described in https://arxiv.org/abs/2002.05202. rms_norm: Use the root mean square layer normalization. multi_query_attention: Use multi-query attention (alias for num_heads_kv=1). num_heads_kv: Number of attention heads for the key and value. head_dim: Number of dimensions per attention head. rotary_dim: Apply rotary embeddings to these first N dimensions. If 0, rotary embeddings are applied to all dimensions. rotary_interleave: Interleave the head dimensions when rotary embeddings are applied. Otherwise the head dimensions are sliced in half. rotary_scaling_type: Type of RoPE scaling. rotary_scaling_factor: Factor used in the RoPE scaling. rotary_base: The base period of the rotary embeddings. sliding_window: Max sequence length to retain in KV Cache. qk_norm: Apply layer normalization to the query and key projections. pre_post_layer_norm: Add post layer norm for each pre norm layer. Nr 5Enabling multi_query_attention implies num_heads_kv=1int16int8Trint32)rrrrrrr"rrrr r!r#r$) ValueErrorrnpdtypetyperrrrrangerEmbeddingsSpec embeddingsscale_embeddingsPositionEncoderSpecposition_encodings LayerNormSpec layer_normrr"TransformerEncoderLayerSpeclayer)selfrrrrrrrrrrrrrrrrrrr r!r"r#r$_s ڞ/tmp/claude-0/-home-danvics-docker-quiz/c1e0577a-e42c-4a3d-b1ea-3edd61103a4e/scratchpad/stt/lib/python3.13/site-packages/ctranslate2/specs/transformer_spec.py__init__TransformerEncoderSpec.__init__ sx !'LA,= KL%:"'*// : ((6*// ; " 0 5 56F G278M2N 2NQK & & (2N !% )@&9&;D # M)77JDO '2'@'@('SD $  %"$((7"3"8"8"HD &:&# "'! ("3(?!)!-%"3$7&;'$7  '#    s 5F* F/) rr1rr8r6rrrr4rr2r")__name__ __module__ __qualname____firstlineno__r ActivationRELUEmbeddingsMergeCONCATintboolrrRotaryScalingTypefloatr<__static_attributes__r;r r s #-8-C-C-H-H%&8C8S8S8Z8Z$)"'(-&+&*"&$("&JN'("(,"'$)1g g g  g  g  ** g  #g &55g "g  g "&g g g  $g smg 3-!g "SM#g $ %g &&n&F&FG'g ( %)g *+g ,! -g .$/g 0"1g g rLr cM\rSrSrS\R R SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS4$S \S \S \S \R S \S\S\S\S\S\S\S\S\S\S\S\S\S\ \S\S\ \ RS\ S\ S\S \S!\S"\S#\S$\S%\ \S&\ \S'\ \S(\ \RS)\ \S*\ \S+\S,\S-\ \S.\4LS/jjr\S05rS1rg)2TransformerDecoderSpecuTFr Nr rrrrrrwith_encoder_attentionrproject_in_outrralignment_layeralignment_headsrralibialibi_use_positive_positions scale_alibirrrr r! original_max_position_embeddingsmax_position_embeddingsparallel_residualshared_layer_normr$rrrr" quant_typequant_group_size quant_bitsr#v_norm external_pre_post_encoder_layersmerged_encoder_attentionc' 8[5UlU(a$U(d [S5eU(a [S5eU(aUbUS:wa [S5eSn[R"S5R U5UlX0l[R"S5R U5Ul[R"S5R U 5Ul [R"S5R U 5Ul [R"5Ul SUl[R UlXlUUlUUlUb*[R"S 5R U5UlU (d U (dU(dUc[-5UlU(a U(d[R0"US 9UlU(a[R0"US 9Ul[R6"5Ul[;U5V's/sHNn'[=S&0S U_S U _S U _SU _SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU#_SU$_SU%_S U&_6PMP sn'UlS!Ul [R Ul!U=(d UU:gURS"'U(a4[R6"5Ul"[R6"5Ul#U (a.U URS#'U"URS$'U!URS%'ggs sn'f)'a Initializes a Transformer decoder specification. Args: num_layers: Number of layers. num_heads: Number of attention heads. pre_norm: Enable the pre-norm Transformer architecture. activation: Activation to apply in the feed-forward network. layernorm_embedding: Apply layer normalization after the embedding layer. with_encoder_attention: Enable the encoder attention sublayers. no_final_norm: Disable the final layer norm in the pre-norm architecture. project_in_out: Add linear transformations after the embedding layer and before the final layer. relative_position: Use relative position representations in the self-attention layers as described in https://arxiv.org/abs/1803.02155. relative_attention_bias: Use relative attention bias in the self-attention layers as described in the T5 paper https://arxiv.org/abs/1910.10683. alignment_layer: Layer index selected for alignment. alignment_heads: Number of attention heads selected for alignment. ffn_glu: Use gated linear units in the FFN layers as described in https://arxiv.org/abs/2002.05202. rms_norm: Use the root mean square layer normalization. alibi: Use attention with linear biases. alibi_use_positive_positions: Use positive positions in the ALiBi definition. scale_alibi: Apply the dot product scale factor to ALiBi. rotary_dim: Apply rotary embeddings to these first N dimensions. If 0, rotary embeddings are applied to all dimensions. rotary_interleave: Interleave the head dimensions when rotary embeddings are applied. Otherwise the head dimensions are sliced in half. rotary_scaling_type: Type of RoPE scaling. rotary_scaling_factor: Factor used in the RoPE scaling. rotary_base: The base period of the rotary embeddings. original_max_position_embeddings: The original max position embeddings for Su rope embeddings max_position_embeddings: The max position embeddings for Su rope embeddings parallel_residual: Use parallel residual connections in each layer block, as used by the GPT-J and GPT-NeoX models. shared_layer_norm: When using parallel residual, share the input and post attention layer norms. pre_post_layer_norm: Add post layer norm for each pre norm layer multi_query_attention: Use multi-query attention (alias for num_heads_kv=1). num_heads_kv: Number of attention heads for the key and value. sliding_window: Max sequence length to retain in KV Cache. quant_type: quantization type used (like awq... for lower bit quantization) quant_group_size: group size of the lower bit quantization quant_bits: number of bit of the quantization (ex: 4bit) external_pre_post_encoder_layers: if the encoder attention pre and processing is done outside the attention. z/The GPT-J block expects a pre-norm architecturez-The GPT-J block does not have cross attentionNr r&r'r(Tr*r)rQrrrrrrrr r!rXrYrZr[r$rrr"r#r_r`raFrquantization_typequantization_bitsquantization_group_sizerK)$dict_configr+r,r-r.rrrrSrTrr0r1r2rOPTIONAL scale_outputsrUrVrWr"r3r4r5r6r LinearSpec projectionr/TransformerDecoderLayerSpecr8start_from_zero_embeddingfinal_logit_softcapping project_in project_out)(r9rrrrrrQrrRrrrSrTrrrUrVrWrrrr r!rXrYrZr[r$rrrr"r\r]r^r#r_r`rar:s( r;r<TransformerDecoderSpec.__init__vs!tv  !RSS% !PQQ 'LA,= KL'*// : ((6*// ;!xx055oF!xx055oF%446 $'00 ,H)&  %"$((7"3"8"8"HD !+"&9&;D # M)77JDO '2'@'@('SD $%0024:&3 2'1 ( '= "3 )@    "   &  #4 %8 '< ( 2R )@ #4 #4 %8  *! ""# $ .% & ' () *2R+ ,*B- 0'3  8*/&'1':':$0E1 I % ,- )446DO*557D  0:DLL, -0:DLL, -6FDLL2 3 M sALcUR$N)rgr9s r;configTransformerDecoderSpec.config"s ||rL)rgrrUrVrTrSr1rnr8r6rrr4rrorprkrWr2rir"rm)r>r?r@rArrBrCrFrGrrrHrI Quantizationr<propertyrurJrKrLr;rNrNus -8-C-C-H-H$)'+#$"'(-! -2!$("&JN'("01'("'"'$)&+&*"&(,9=*.$(;@).OjGjGjG jG  ** jG " jG!%jGjGjG jG"&jGjGjGjGjG !jG"'+#jG$%jG&SM'jG( )jG*&n&F&FG+jG, %-jG./jG0+.1jG2"%3jG4 5jG6 7jG8"9jG: $;jG<sm=jG>3-?jG@! AjGB[556CjGD#3-EjGFSMGjGHIjGJKjGL+34.MjGN#'OjGXrLrNc v\rSrSrS S\\S\S\\RS\ S\ S\4 S jjr S r g) r7i'Nrrrr r!r$c[R"SUUUUUUUU U U U U S9 Ul[X4S9UlU(a[ R "US9Ul[ R "US9Ul[ R "US9Ul [ R "US9Ul [URS5 [URS5 gg)NT) self_attentionrrrrrr"rrrr r!r#glurr)r6) rMultiHeadAttentionSpecr{FeedForwardSpecffnrr5input_layer_normpost_attention_layer_normpre_feedforward_layer_normpost_feedforward_layer_normdelattr)r9rrrrrrr"rrrr r!r#r$s r;r<$TransformerEncoderLayerSpec.__init__(s"-CC/$;%)!/ 3"7# #wB $/$=$=x$PD !-8-F-F!.D */:.G.G!/D +0;/H/H!0D , D'' 6 DHHl + rL)rrrrrr{)FFFFNNNNTNr r FF) r>r?r@rArrFrGrrHrIr<rJrKrLr;r7r7's %$("&JN'("$)/,SM/, /,&n&F&FG/, %/,/,"/,/,rLr7cJ\rSrSrSSjrSrg)rliZNc [R"S0SS_SU_SU_SU_SU_SU_SU_S U _S U _S U _S U _S U_SU_SU_SU_SU_SU_6UlU(a'U(d [R"UUUUUUSLS9Ul[ XES9UlU (aU(a[ R"5UlO4[ R"5Ul [ R"5Ul [URS5 [UR S5 U(a[ R"US9Ul [ R"US9Ul U(a9U(a2[ R"US9Ul [ R"US9Ul [ R"US9Ul[ R"US9Ul[URS5 [UR S5 [ R"Ulg)Nr{Trrrrrrr r!rXrYrrr"r#r_raF)rrrr"r#has_normr|r6r)rK)rr~r{ attentionrrrr5r[rrr*external_post_encoder_attention_layer_norm)external_pre_encoder_attention_layer_normrrrrh layer_scalar)r9rQrrrrrrrr r!rXrYrZr[r$rrr"r#r_r`ras r;r<$TransformerDecoderLayerSpec.__init__[s2-CC  / %<   "  0  !4 #8 $ .N %< &  *  ! "&># ( "*B+BB!)!-9UB DN#wB  )4)B)B)D&(3(A(A(C%1<1J1J1L. D'' 6 DHHl + $/$=$=x$PD !-8-F-F!.D *&*J--x@? --x@> /:.G.G!/D +0;/H/H!0D , D'' 6 DHHl +&//rL) rrrrrrrrrr{r[)TFFFFNTNr r rrFFFNNNFFFFr>r?r@rAr<rJrKrLr;rlrlZsL $ % )* !!).!&/]0rLrlc\rSrSrSSjrSrg)ric[R"US9Ul[R"5Ul[R"5UlU(a"5Ulgg)Nr))rr5r6rjlinear_0linear_1linear_0_noact)r9r}rs r;r<FeedForwardSpec.__init__sM%33XF#..0 #..0 "-"8"8":D  rL)r6rrrN)FFrrKrLr;rrs;rLrc\rSrSrSrSrg)r3ic.[RUlgrs)rrh encodingsrts r;r<PositionEncoderSpec.__init__s#,,rL)rNrrKrLr;r3r3s-rLr3c>^\rSrSrSrSS\\4U4SjjjrSrU=r $)TransformerConfigiz%Configuration for Transformer models.layer_norm_epsilonc *>[TU]"SSU0UD6 g)zInitializes the configuration for Transformer models. Args: layer_norm_epsilon: The layer norm epsilon value. **kwargs: Additional configuration. rNrKsuperr<r9rkwargs __class__s r;r<TransformerConfig.__init__ I,>I&IrLrKrs r>r?r@rA__doc__rrIr<rJ __classcell__rs@r;rrs/J8E?JJrLrc d^\rSrSrSrS\S\4U4Sjjr\SSS\ RRSS S \ RRSSSSS4 S \\\\\44S \S \S \S\S\ RS\S\S\S\ RS\S\S\S\S\4Sjj5r\S5r\S5rSrSrSrSrU=r$) TransformerSpecizDescribes a Transformer model. The specification is invariant to hidden dimensions but requires to explicitly set the number of layers and attention heads. encoderdecoderc>[U[5(d [S5e[U[5(d [S5e[TU]5 XlX lURRSUR R5 g)zInitializes a Transformer model specification. Args: encoder: The encoder specification. decoder: The decoder specification. 1encoder argument must be a TransformerEncoderSpec1decoder argument must be a TransformerDecoderSpecrN) isinstancer TypeErrorrNrr<rrrg add_attributer)r9rrrs r;r<TransformerSpec.__init__so'#9::OP P'#9::OP P    "" #T\\%G%G rLFTrPr rrwith_relative_positionrrrrSrTrrrrrrrc[U[[45(aUunnOXnn[UUUUUU U U UU U UUS9 n[ UUUUUU UU UUU UUS9 nU"UU5$)aCreates a Transformer model specification. Args: num_layers: Number of encoder and decoder layers, or a 2-tuple if the number is different. num_heads: Number of attention heads. with_relative_position: Use relative position representations in the self-attention layers as described in https://arxiv.org/abs/1803.02155. pre_norm: Enable the pre-norm Transformer architecture. no_final_norm: Disable the final layer norm in the pre-norm architecture. activation: Activation to apply in the feed-forward network. alignment_layer: Layer index selected for alignment. alignment_heads: Number of attention heads selected for alignment. num_source_embeddings: Number of source embeddings. embeddings_merge: When :obj:`num_source_embeddings` > 1, specify how the embeddings are merged. layernorm_embedding: Apply layer normalization after the embedding layer. relative_attention_bias: Use relative attention bias in the self-attention layers as described in the T5 paper https://arxiv.org/abs/1910.10683. ffn_glu: Use gated linear units in the FFN layer as described in https://arxiv.org/abs/2002.05202. rms_norm: Use the root mean square layer normalization. multi_query_attention: Use multi-query attention. ) rrrrrrrrrrr) rrrrrrrSrTrrr)rlisttupler rN)clsrrrrrrrSrTrrrrrrrnum_encoder_layersnum_decoder_layersrrs r; from_configTransformerSpec.from_configsV j4- 0 05? 2  25? 2 (  '!"7- 34$;"7  )  '! 34$;++"7  7G$$rLcg)NrrKrts r;nameTransformerSpec.nameDs rLcg)NrKrts r;revisionTransformerSpec.revisionHrLc[5$rs)rrts r;get_default_config"TransformerSpec.get_default_configLs  ""rLcURRVs/sHoRRSPM sn$s snfNrrr1weightshape)r9specs r;get_source_vocabulary_size*TransformerSpec.get_source_vocabulary_sizeOs3151H1HI1H !!!$1HIIIs">c\URRRRS$rrr1rrrts r;get_target_vocabulary_size*TransformerSpec.get_target_vocabulary_sizeR#||&&--33A66rL)rr)r>r?r@rArr rNr< classmethodrrBrCrDrErrFrrGrrxrrrrrrJrrs@r;rrss  - 8N * (-#-8-C-C-H-H! %&8C8S8S8Z8Z$)(-&+!O%#uS#X./O%O%!% O%  O%  O% **O%O%O% #O%&55O%"O%"&O%O%O% $!O%O%b!!#J77rLrc>^\rSrSrSrSS\\4U4SjjjrSrU=r $)TransformerDecoderModelConfigiVz-Configuration for Transformer decoder models.rc *>[TU]"SSU0UD6 g)zInitializes the configuration for Transformer decoder models. Args: layer_norm_epsilon: The layer norm epsilon value. **kwargs: Additional configuration. rNrKrrs r;r<&TransformerDecoderModelConfig.__init__YrrLrKrsrrs@r;rrV7J8E?JJrLrcB^\rSrSrSrS\4U4Sjjr\S\RRSSSSSSSSSSSSSS S S SSSSSSSSSSSS4S \ S \ S \ S\RS\ S\ S\ S\ S\ S\ S\ S\ S\ S\ \ S\ S\ \RS\S\S\ S\ S\ S \ S!\ S"\ S#\ \ S$\ \ S%\ \ S&\ \R"S'\ \ S(\ \ S)\ S*\ 4@S+jj5r\S,5r\S-5rS.rS/rS0rU=r$)1TransformerDecoderModelSpecicz3Describes a Transformer decoder model (e.g. GPT-2).rc>[U[5(d [S5e[TU]5 XlUR R R5H up#URRX#5 M" g)zdInitializes a Transformer decoder model specification. Args: decoder: The decoder specification. rN) rrNrrr<rruitemsrgr)r9rkeyvaluers r;r<$TransformerDecoderModelSpec.__init__fs` '#9::OP P  ,,--335JC LL & &s 26rLTFNr r rrrrrrrrRrrrrUrVrWrrrr r!rXrYrZr[r$rrrr"r\r]r^r#r_c![UU40SU_SU_SU_SS_SU_SU_SU_S U _S U _S U _S U _S U _SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_S U _6n!U"U!5$)!aCreates a Transformer decoder model specification. Args: num_layers: Number of decoder layers. num_heads: Number of attention heads. pre_norm: Enable the pre-norm Transformer architecture. activation: Activation to apply in the feed-forward network. layernorm_embedding: Apply layer normalization after the embedding layer. no_final_norm: Do not apply layer normalization after the last decoder block. project_in_out: Add a linear layer after the embedding layer and another one before the final output projection. with_relative_position: Enable relative position representations modules. ffn_glu: Use gated linear units in the FFN layers as described in https://arxiv.org/abs/2002.05202. rms_norm: Use the root mean square layer normalization. alibi: Use attention with linear biases. alibi_use_positive_positions: Use positive positions in the ALiBi definition. scale_alibi: Apply the dot product scale factor to ALiBi. rotary_dim: Apply rotary embeddings to these first N dimensions. If 0, rotary embeddings are applied to all dimensions. rotary_interleave: Interleave the head dimensions when rotary embeddings are applied. Otherwise the head dimensions are sliced in half. rotary_scaling_type: Type of RoPE scaling. rotary_scaling_factor: Factor used in the RoPE scaling. rotary_base: The base period of the rotary embeddings. original_max_position_embeddings: The original max position embeddings for Su rope embeddings max_position_embeddings: The max position embeddings for Su rope embeddings parallel_residual: Use parallel residual connections in each layer block, as used by the GPT-J and GPT-NeoX models. shared_layer_norm: When using parallel residual, share the input and post attention layer norms. pre_post_layer_norm: add post layer norm for each pre norm layer multi_query_attention: Use multi-query attention (alias for num_heads_kv=1). num_heads_kv: Number of attention heads for the key and value. head_dim: Number of head sliding_window: max sequence length to retain KV cache quant_type: quantization type used (like awq... for lower bit quantization) quant_group_size: group size of the lower bit quantization quant_bits: number of bit of the quantization (ex: 4bit) rrrrQFrrRrrrrUrVrWrrrr r!rXrYrZr[r$rrrr"r\r]r^r#r_)rN)"rrrrrrrrRrrrrUrVrWrrrr r!rXrYrZr[r$rrrr"r\r]r^r#r_rs" r;r'TransformerDecoderModelSpec.from_configts^Z)  " " " " !4 " $) " (" *" 5" " " " *F" $" "" 0!" "!4#" $#8%" &$'" (.N)" *%<+" ,0-" .0/" 0!41" 2#83" 4&5" 67" 8*9" :";" <.=" >"?" @A" BC" H7|rLcg)NrNrKrts r;r TransformerDecoderModelSpec.name'rLcg)NrKrts r;r$TransformerDecoderModelSpec.revisionrrLc[5$rs)rrts r;r.TransformerDecoderModelSpec.get_default_config ,..rLc\URRRRS$rrrts r;get_vocabulary_size/TransformerDecoderModelSpec.get_vocabulary_sizerrL)r)r>r?r@rArrNr<rrrBrCrFrGrrrHrIrwrrxrrrrrJrrs@r;rrcs`= 3 6 3 -8-C-C-H-H$)#$',-2!$("&JN'("01'("'"'$)&+&*"&(,9=*.$(Cppp p  ** p " ppp!%pppp'+ppSMp  !p"&n&F&FG#p$ %%p&'p(+.)p*"%+p, -p. /p0"1p2 $3p4sm5p63-7p8! 9p:[556;p<#3-=p>SM?p@ApBCppd((/77rLrc>^\rSrSrSrSS\\4U4SjjjrSrU=r $)TransformerEncoderModelConfigiz-Configuration for Transformer encoder models.rc *>[TU]"SSU0UD6 g)zInitializes the configuration for Transformer encoder models. Args: layer_norm_epsilon: The layer norm epsilon value. **kwargs: Additional configuration. rNrKrrs r;r<&TransformerEncoderModelConfig.__init__rrLrKrsrrs@r;rrrrLrc^\rSrSrSrS\R R4S\S\ S\R 4U4Sjjjr \ S5r \ S 5r S rS rS rU=r$) TransformerEncoderModelSpeciz2Describes a Transformer encoder model (e.g. BERT).Fr pooling_layerpooling_activationcf>[U[5(d [S5e[TU]5 XlUR RSUR R5 U(aE[R"5Ul [R"S5RU5Ulgg)zInitializes a Transformer encoder model specification. Args: encoder: The encoder specification. pooling_layer: Add the pooling layer. pooling_activation: The activation to apply after the pooling layer. rrr(N)rr rrr<rrgrrrrj pooler_denser,r-r.pooler_activation)r9rrrrs r;r<$TransformerEncoderModelSpec.__init__s'#9::OP P   "" #T\\%G%G   + 6 6 8D %'XXf%5%:%:;M%ND " rLcg)Nr rKrts r;r TransformerEncoderModelSpec.name rrLcg)Nr rKrts r;r$TransformerEncoderModelSpec.revision$rrLc[5$rs)rrts r;r.TransformerEncoderModelSpec.get_default_config(rrLcbURRSRRS$rrrts r;r/TransformerEncoderModelSpec.get_vocabulary_size+s(||&&q)0066q99rL)rrr)r>r?r@rArrrBTanhr rGr<rxrrrrrJrrs@r;rrs< $5@5K5K5P5P O'OO(22 OO4((/::rLr)rtypingrrrnumpyr,ctranslate2.specsrrr LayerSpecr rNr7rlrr3SequenceToSequenceModelConfigrSequenceToSequenceModelSpecrLanguageModelConfigrLanguageModelSpecrrrrKrLr;r s6))EEh Z11h VoZ11od0,*"6"60,f^0*"6"6^0B;j**;-*..- J @@ J}7j<<}7@ JJ$B$B JP7*">">P7f JJ$B$B J):*">">):rL