alihan/datamodel-code-generator
1
0
Fork 0
mirror of https://github.com/koxudaxi/datamodel-code-generator.git synced 2024-03-18 14:54:37 +03:00
Code Issues Packages Projects Releases Wiki Activity

Add --use-default-kwarg option for better type checker compatibility (#1034)

Browse Source 
* Add --use-default-kwarg option for better type checker compatibility

Python 3.11 introduced the `@dataclass_transform` decorator, and older
verisions of python can access the same from the `typing_extensions`
package. This decorator lets type checkers infer the right `__init__`
function for "dataclass-like" classes. It relies on some conventional
keyword arguments to determine whether a given field is a required
argument for `__init__`. Essentially, if the `default=` kwarg is
present, then the field is not required.

Pydantic allows for the `default` argument to be positional, but this
doesn't give the right hints to the type checker. If we set `default` as
a kwarg for fields that do have defaults, then the signature of
`__init__` is inferred correctly.

Example generated code:

```python

# Without --use-default-kwarg: type checkers will assume `b` and `c` are
# required arguments to `__init__`
class Model(BaseModel):
    a: str = Field(..., description='Required field')
    b: Optional[str] = Field(None, description='Optional field')
    c: str = Field("default", description='Another optional field')

# With --use-default-kwarg: type checkers will correctly assume `b` and
# `c` are optional arguments to `__init__`
class Model(BaseModel):
    a: str = Field(..., description='Required field')
    b: Optional[str] = Field(default=None, description='Optional field')
    c: str = Field(default="default", description='Another optional field')
```

* Update README/docs with output of `datamodel-codegen -h`

Co-authored-by: Zack Yancey <mail@zackyancey.com>
This commit is contained in:
Zack Yancey
2023-01-19 16:31:21 -07:00
committed by GitHub
parent 34079734bb
commit 45a5031029
11 changed files with 197 additions and 46 deletions
Download Patch File Download Diff File Expand all files Collapse all files

61
README.md
View File

@@ -24,7 +24,7 @@ To install `datamodel-code-generator`:
$ pip install datamodel-code-generator
```
## Simple usage
## Simple usage
You can generate models from a local file.
```bash
$ datamodel-codegen --input api.yaml --output model.py
@@ -236,7 +236,7 @@ class Apis(BaseModel):
## Which project uses it?
These OSS use datamodel-code-generator to generate many models. We can learn about use-cases from these projects.
- [Netflix/consoleme](https://github.com/Netflix/consoleme)
- [Netflix/consoleme](https://github.com/Netflix/consoleme)
- *[How do I generate models from the Swagger specification?](https://github.com/Netflix/consoleme/blob/master/docs/gitbook/faq.md#how-do-i-generate-models-from-the-swagger-specification)*
- [DataDog/integrations-core](https://github.com/DataDog/integrations-core)
- *[Config models](https://github.com/DataDog/integrations-core/blob/master/docs/developer/meta/config-models.md)*
@@ -293,7 +293,7 @@ usage: datamodel-codegen [-h] [--input INPUT] [--url URL]
[--http-headers HTTP_HEADER [HTTP_HEADER ...]]
[--http-ignore-tls]
[--input-file-type {auto,openapi,jsonschema,json,yaml,dict,csv}]
[--openapi-scopes {schemas,paths} [{schemas,paths} ...]]
[--openapi-scopes {schemas,paths,tags} [{schemas,paths,tags} ...]]
[--output OUTPUT] [--base-class BASE_CLASS]
[--field-constraints] [--use-annotated]
[--use_non_positive_negative_number_constrained_types]
@@ -303,30 +303,34 @@ usage: datamodel-codegen [-h] [--input INPUT] [--url URL]
[--strip-default-none]
[--disable-appending-item-suffix]
[--allow-population-by-field-name]
[--enable-faux-immutability] [--use-default]
[--force-optional] [--strict-nullable]
[--allow-extra-fields] [--enable-faux-immutability]
[--use-default] [--force-optional]
[--strict-nullable]
[--strict-types {str,bytes,int,float,bool} [{str,bytes,int,float,bool} ...]]
[--disable-timestamp] [--use-standard-collections]
[--use-generic-container-types]
[--use-schema-description] [--use-field-description] [--reuse-model]
[--collapse-root-models] [--enum-field-as-literal {all,one}]
[--use-union-operator] [--use-schema-description]
[--use-field-description] [--use-default-kwarg]
[--reuse-model] [--collapse-root-models]
[--enum-field-as-literal {all,one}]
[--set-default-enum-member]
[--empty-enum-field-name EMPTY_ENUM_FIELD_NAME]
[--capitalise-enum-members]
[--special-field-name-prefix SPECIAL_FIELD_NAME_PREFIX]
[--use-subclass-enum]
[--class-name CLASS_NAME] [--use-title-as-name]
[--use-subclass-enum] [--class-name CLASS_NAME]
[--use-title-as-name]
[--custom-template-dir CUSTOM_TEMPLATE_DIR]
[--extra-template-data EXTRA_TEMPLATE_DATA]
[--aliases ALIASES]
[--target-python-version {3.6,3.7,3.8,3.9}]
[--target-python-version {3.6,3.7,3.8,3.9,3.10,3.11}]
[--wrap-string-literal] [--validation]
[--encoding ENCODING] [--debug] [--version]
[--use-double-quotes] [--encoding ENCODING] [--debug]
[--version]
optional arguments:
options:
-h, --help show this help message and exit
--input INPUT Input file/directory (default: stdin)
--url URL Input file URL. `--input` is ignore when `--url` is
--url URL Input file URL. `--input` is ignored when `--url` is
used
--http-headers HTTP_HEADER [HTTP_HEADER ...]
Set headers in HTTP requests to the remote host.
@@ -335,7 +339,7 @@ optional arguments:
certificate
--input-file-type {auto,openapi,jsonschema,json,yaml,dict,csv}
Input file type (default: auto)
--openapi-scopes {schemas,paths} [{schemas,paths} ...]
--openapi-scopes {schemas,paths,tags} [{schemas,paths,tags} ...]
Scopes of OpenAPI model generation (default: schemas)
--output OUTPUT Output file (default: stdout)
--base-class BASE_CLASS
@@ -344,22 +348,25 @@ optional arguments:
--use-annotated Use typing.Annotated for Field(). Also, `--field-
constraints` option will be enabled.
--use_non_positive_negative_number_constrained_types
Use the Non{Positive,Negative}{FloatInt} types instead of the corresponding con*
constrained types.
Use the Non{Positive,Negative}{FloatInt} types instead
of the corresponding con* constrained types.
--field-extra-keys FIELD_EXTRA_KEYS [FIELD_EXTRA_KEYS ...]
Add extra keys to field parameters
--field-include-all-keys
Add all keys to field parameters
--snake-case-field Change camel-case field name to snake-case
--original-field-name-delimiter ORIGINAL_FIELD_NAME_DELIMITER
Set delimiter to convert to snake case. This option only
can be used with --snake-case-field (default: `_` )
Set delimiter to convert to snake case. This option
only can be used with --snake-case-field (default: `_`
)
--strip-default-none Strip default None on fields
--disable-appending-item-suffix
Disable appending `Item` suffix to model name in an
array
--allow-population-by-field-name
Allow population by field name
--allow-extra-fields Allow to pass extra fields, if this flag is not
passed, extra fields are forbidden.
--enable-faux-immutability
Enable faux immutability
--use-default Use default value even if a field is required
@@ -376,15 +383,18 @@ optional arguments:
(typing.Sequence, typing.Mapping). If `--use-standard-
collections` option is set, then import from
collections.abc instead of typing
--use-union-operator Use | operator for Union type (PEP 604).
--use-schema-description
Use schema description to populate class docstring
--use-field-description
Use schema description to populate field docstring
--use-default-kwarg Use `default=` instead of a positional argument for
Fields that have default values.
--reuse-model Re-use models on the field when a module has the model
with the same content
--collapse-root-models
Models generated with a root-type field will be
merged into the models using that root-type model
mergedinto the models using that root-type model
--enum-field-as-literal {all,one}
Parse enum field as literal. all: all enum field type
are Literal. one: field type is Literal when an enum
@@ -396,8 +406,10 @@ optional arguments:
--capitalise-enum-members
Capitalize field names on enum
--special-field-name-prefix SPECIAL_FIELD_NAME_PREFIX
--use-subclass-enum Define Enum class as subclass with field type when enum has
type (int, float, bytes, str)
Set field name prefix when first character can't be
used as Python field name (default: `field`)
--use-subclass-enum Define Enum class as subclass with field type when
enum has type (int, float, bytes, str)
--class-name CLASS_NAME
Set class name of root model
--use-title-as-name use titles as class names of models
@@ -406,14 +418,17 @@ optional arguments:
--extra-template-data EXTRA_TEMPLATE_DATA
Extra template data
--aliases ALIASES Alias mapping file
--target-python-version {3.6,3.7,3.8,3.9}
--target-python-version {3.6,3.7,3.8,3.9,3.10,3.11}
target python version (default: 3.7)
--wrap-string-literal
Wrap string literal by using black `experimental-
string-processing` option (require black 20.8b0 or
later)
--validation Enable validation (Only OpenAPI)
--encoding ENCODING The encoding of input and output (default: UTF-8)
--use-double-quotes Model generated with double quotes. Single quotes or
your black config skip_string_normalization value will
be used without this option.
--encoding ENCODING The encoding of input and output (default: cp1252)
--debug show debug message
--version show version
```

2
datamodel_code_generator/__init__.py
View File

@@ -222,6 +222,7 @@ def generate(
use_standard_collections: bool = False,
use_schema_description: bool = False,
use_field_description: bool = False,
use_default_kwarg: bool = False,
reuse_model: bool = False,
encoding: str = 'utf-8',
enum_field_as_literal: Optional[LiteralType] = None,
@@ -351,6 +352,7 @@ def generate(
else None,
use_schema_description=use_schema_description,
use_field_description=use_field_description,
use_default_kwarg=use_default_kwarg,
reuse_model=reuse_model,
enum_field_as_literal=enum_field_as_literal,
set_default_enum_member=set_default_enum_member,

8
datamodel_code_generator/__main__.py
View File

@@ -258,6 +258,12 @@ arg_parser.add_argument(
default=None,
)
arg_parser.add_argument(
'--use-default-kwarg',
action='store_true',
help="Use `default=` instead of a positional argument for Fields that have default values.",
)
arg_parser.add_argument(
'--reuse-model',
help='Re-use models on the field when a module has the model with the same content',
@@ -495,6 +501,7 @@ class Config(BaseModel):
use_standard_collections: bool = False
use_schema_description: bool = False
use_field_description: bool = False
use_default_kwarg: bool = True
reuse_model: bool = False
encoding: str = DEFAULT_ENCODING
enum_field_as_literal: Optional[LiteralType] = None
@@ -645,6 +652,7 @@ def main(args: Optional[Sequence[str]] = None) -> Exit:
use_standard_collections=config.use_standard_collections,
use_schema_description=config.use_schema_description,
use_field_description=config.use_field_description,
use_default_kwarg=config.use_default_kwarg,
reuse_model=config.reuse_model,
encoding=config.encoding,
enum_field_as_literal=config.enum_field_as_literal,

1
datamodel_code_generator/model/base.py
View File

@@ -63,6 +63,7 @@ class DataModelFieldBase(_BaseModel):
use_field_description: bool = False
const: bool = False
original_name: Optional[str] = None
use_default_kwarg: bool = False
_exclude_fields: ClassVar[Set[str]] = {'parent'}
_pass_fields: ClassVar[Set[str]] = {'parent', 'data_type'}

5
datamodel_code_generator/model/pydantic/base_model.py
View File

@@ -69,6 +69,11 @@ class DataModelField(DataModelFieldBase):
def field(self) -> Optional[str]:
"""for backwards compatibility"""
result = str(self)
if self.use_default_kwarg and not result.startswith("Field(..."):
# Use `default=` for fields that have a default value so that type
# checkers using @dataclass_transform can infer the field as
# optional in __init__.
result = result.replace("Field(", "Field(default=")
if result == "":
return None

2
datamodel_code_generator/parser/base.py
View File

@@ -330,6 +330,7 @@ class Parser(ABC):
base_path: Optional[Path] = None,
use_schema_description: bool = False,
use_field_description: bool = False,
use_default_kwarg: bool = False,
reuse_model: bool = False,
encoding: str = 'utf-8',
enum_field_as_literal: Optional[LiteralType] = None,
@@ -390,6 +391,7 @@ class Parser(ABC):
)
self.use_schema_description: bool = use_schema_description
self.use_field_description: bool = use_field_description
self.use_default_kwarg: bool = use_default_kwarg
self.reuse_model: bool = reuse_model
self.encoding: str = encoding
self.enum_field_as_literal: Optional[LiteralType] = enum_field_as_literal

3
datamodel_code_generator/parser/jsonschema.py
View File

@@ -339,6 +339,7 @@ class JsonSchemaParser(Parser):
base_path: Optional[Path] = None,
use_schema_description: bool = False,
use_field_description: bool = False,
use_default_kwarg: bool = False,
reuse_model: bool = False,
encoding: str = 'utf-8',
enum_field_as_literal: Optional[LiteralType] = None,
@@ -393,6 +394,7 @@ class JsonSchemaParser(Parser):
base_path=base_path,
use_schema_description=use_schema_description,
use_field_description=use_field_description,
use_default_kwarg=use_default_kwarg,
reuse_model=reuse_model,
encoding=encoding,
enum_field_as_literal=enum_field_as_literal,
@@ -498,6 +500,7 @@ class JsonSchemaParser(Parser):
extras={**self.get_field_extras(field)},
use_annotated=self.use_annotated,
use_field_description=self.use_field_description,
use_default_kwarg=self.use_default_kwarg,
original_name=original_field_name,
)

2
datamodel_code_generator/parser/openapi.py
View File

@@ -168,6 +168,7 @@ class OpenAPIParser(JsonSchemaParser):
base_path: Optional[Path] = None,
use_schema_description: bool = False,
use_field_description: bool = False,
use_default_kwarg: bool = False,
reuse_model: bool = False,
encoding: str = 'utf-8',
enum_field_as_literal: Optional[LiteralType] = None,
@@ -223,6 +224,7 @@ class OpenAPIParser(JsonSchemaParser):
base_path=base_path,
use_schema_description=use_schema_description,
use_field_description=use_field_description,
use_default_kwarg=use_default_kwarg,
reuse_model=reuse_model,
encoding=encoding,
enum_field_as_literal=enum_field_as_literal,

61
docs/index.md
View File

@@ -24,7 +24,7 @@ To install `datamodel-code-generator`:
$ pip install datamodel-code-generator
```
## Simple usage
## Simple usage
You can generate models from a local file.
```bash
$ datamodel-codegen --input api.yaml --output model.py
@@ -236,7 +236,7 @@ class Apis(BaseModel):
## Which project uses it?
These OSS use datamodel-code-generator to generate many models. We can learn about use-cases from these projects.
- [Netflix/consoleme](https://github.com/Netflix/consoleme)
- [Netflix/consoleme](https://github.com/Netflix/consoleme)
- *[How do I generate models from the Swagger specification?](https://github.com/Netflix/consoleme/blob/master/docs/gitbook/faq.md#how-do-i-generate-models-from-the-swagger-specification)*
- [DataDog/integrations-core](https://github.com/DataDog/integrations-core)
- *[Config models](https://github.com/DataDog/integrations-core/blob/master/docs/developer/meta/config-models.md)*
@@ -293,7 +293,7 @@ usage: datamodel-codegen [-h] [--input INPUT] [--url URL]
[--http-headers HTTP_HEADER [HTTP_HEADER ...]]
[--http-ignore-tls]
[--input-file-type {auto,openapi,jsonschema,json,yaml,dict,csv}]
[--openapi-scopes {schemas,paths} [{schemas,paths} ...]]
[--openapi-scopes {schemas,paths,tags} [{schemas,paths,tags} ...]]
[--output OUTPUT] [--base-class BASE_CLASS]
[--field-constraints] [--use-annotated]
[--use_non_positive_negative_number_constrained_types]
@@ -303,30 +303,34 @@ usage: datamodel-codegen [-h] [--input INPUT] [--url URL]
[--strip-default-none]
[--disable-appending-item-suffix]
[--allow-population-by-field-name]
[--enable-faux-immutability] [--use-default]
[--force-optional] [--strict-nullable]
[--allow-extra-fields] [--enable-faux-immutability]
[--use-default] [--force-optional]
[--strict-nullable]
[--strict-types {str,bytes,int,float,bool} [{str,bytes,int,float,bool} ...]]
[--disable-timestamp] [--use-standard-collections]
[--use-generic-container-types]
[--use-schema-description] [--use-field-description] [--reuse-model]
[--collapse-root-models] [--enum-field-as-literal {all,one}]
[--use-union-operator] [--use-schema-description]
[--use-field-description] [--use-default-kwarg]
[--reuse-model] [--collapse-root-models]
[--enum-field-as-literal {all,one}]
[--set-default-enum-member]
[--empty-enum-field-name EMPTY_ENUM_FIELD_NAME]
[--capitalise-enum-members]
[--special-field-name-prefix SPECIAL_FIELD_NAME_PREFIX]
[--use-subclass-enum]
[--class-name CLASS_NAME] [--use-title-as-name]
[--use-subclass-enum] [--class-name CLASS_NAME]
[--use-title-as-name]
[--custom-template-dir CUSTOM_TEMPLATE_DIR]
[--extra-template-data EXTRA_TEMPLATE_DATA]
[--aliases ALIASES]
[--target-python-version {3.6,3.7,3.8,3.9}]
[--target-python-version {3.6,3.7,3.8,3.9,3.10,3.11}]
[--wrap-string-literal] [--validation]
[--encoding ENCODING] [--debug] [--version]
[--use-double-quotes] [--encoding ENCODING] [--debug]
[--version]
optional arguments:
options:
-h, --help show this help message and exit
--input INPUT Input file/directory (default: stdin)
--url URL Input file URL. `--input` is ignore when `--url` is
--url URL Input file URL. `--input` is ignored when `--url` is
used
--http-headers HTTP_HEADER [HTTP_HEADER ...]
Set headers in HTTP requests to the remote host.
@@ -335,7 +339,7 @@ optional arguments:
certificate
--input-file-type {auto,openapi,jsonschema,json,yaml,dict,csv}
Input file type (default: auto)
--openapi-scopes {schemas,paths} [{schemas,paths} ...]
--openapi-scopes {schemas,paths,tags} [{schemas,paths,tags} ...]
Scopes of OpenAPI model generation (default: schemas)
--output OUTPUT Output file (default: stdout)
--base-class BASE_CLASS
@@ -344,22 +348,25 @@ optional arguments:
--use-annotated Use typing.Annotated for Field(). Also, `--field-
constraints` option will be enabled.
--use_non_positive_negative_number_constrained_types
Use the Non{Positive,Negative}{FloatInt} types instead of the corresponding con*
constrained types.
Use the Non{Positive,Negative}{FloatInt} types instead
of the corresponding con* constrained types.
--field-extra-keys FIELD_EXTRA_KEYS [FIELD_EXTRA_KEYS ...]
Add extra keys to field parameters
--field-include-all-keys
Add all keys to field parameters
--snake-case-field Change camel-case field name to snake-case
--original-field-name-delimiter ORIGINAL_FIELD_NAME_DELIMITER
Set delimiter to convert to snake case. This option only
can be used with --snake-case-field (default: `_` )
Set delimiter to convert to snake case. This option
only can be used with --snake-case-field (default: `_`
)
--strip-default-none Strip default None on fields
--disable-appending-item-suffix
Disable appending `Item` suffix to model name in an
array
--allow-population-by-field-name
Allow population by field name
--allow-extra-fields Allow to pass extra fields, if this flag is not
passed, extra fields are forbidden.
--enable-faux-immutability
Enable faux immutability
--use-default Use default value even if a field is required
@@ -376,15 +383,18 @@ optional arguments:
(typing.Sequence, typing.Mapping). If `--use-standard-
collections` option is set, then import from
collections.abc instead of typing
--use-union-operator Use | operator for Union type (PEP 604).
--use-schema-description
Use schema description to populate class docstring
--use-field-description
Use schema description to populate field docstring
--use-default-kwarg Use `default=` instead of a positional argument for
Fields that have default values.
--reuse-model Re-use models on the field when a module has the model
with the same content
--collapse-root-models
Models generated with a root-type field will be
merged into the models using that root-type model
mergedinto the models using that root-type model
--enum-field-as-literal {all,one}
Parse enum field as literal. all: all enum field type
are Literal. one: field type is Literal when an enum
@@ -396,8 +406,10 @@ optional arguments:
--capitalise-enum-members
Capitalize field names on enum
--special-field-name-prefix SPECIAL_FIELD_NAME_PREFIX
--use-subclass-enum Define Enum class as subclass with field type when enum has
type (int, float, bytes, str)
Set field name prefix when first character can't be
used as Python field name (default: `field`)
--use-subclass-enum Define Enum class as subclass with field type when
enum has type (int, float, bytes, str)
--class-name CLASS_NAME
Set class name of root model
--use-title-as-name use titles as class names of models
@@ -406,14 +418,17 @@ optional arguments:
--extra-template-data EXTRA_TEMPLATE_DATA
Extra template data
--aliases ALIASES Alias mapping file
--target-python-version {3.6,3.7,3.8,3.9}
--target-python-version {3.6,3.7,3.8,3.9,3.10,3.11}
target python version (default: 3.7)
--wrap-string-literal
Wrap string literal by using black `experimental-
string-processing` option (require black 20.8b0 or
later)
--validation Enable validation (Only OpenAPI)
--encoding ENCODING The encoding of input and output (default: UTF-8)
--use-double-quotes Model generated with double quotes. Single quotes or
your black config skip_string_normalization value will
be used without this option.
--encoding ENCODING The encoding of input and output (default: cp1252)
--debug show debug message
--version show version
```

74
tests/data/expected/main/main_use_default_kwarg/output.py Normal file
View File

@@ -0,0 +1,74 @@
# generated by datamodel-codegen:
# filename: nullable.yaml
# timestamp: 2019-07-26T00:00:00+00:00
from __future__ import annotations
from typing import List, Optional
from pydantic import AnyUrl, BaseModel, Field
class Cursors(BaseModel):
prev: str
next: Optional[str] = 'last'
index: float
tag: Optional[str] = None
class TopLevel(BaseModel):
cursors: Cursors
class Info(BaseModel):
name: str
class User(BaseModel):
info: Info
class Api(BaseModel):
apiKey: Optional[str] = Field(
default=None, description='To be used as a dataset parameter value'
)
apiVersionNumber: Optional[str] = Field(
default=None, description='To be used as a version parameter value'
)
apiUrl: Optional[AnyUrl] = Field(
default=None, description="The URL describing the dataset's fields"
)
apiDocumentationUrl: Optional[AnyUrl] = Field(
default=None, description='A URL to the API console for each API'
)
class Apis(BaseModel):
__root__: Optional[List[Api]] = None
class EmailItem(BaseModel):
author: str
address: str = Field(..., description='email address')
description: Optional[str] = 'empty'
tag: Optional[str] = None
class Email(BaseModel):
__root__: List[EmailItem]
class Id(BaseModel):
__root__: int
class Description(BaseModel):
__root__: Optional[str] = 'example'
class Name(BaseModel):
__root__: Optional[str] = None
class Tag(BaseModel):
__root__: str

24
tests/test_main.py
View File

@@ -4515,3 +4515,27 @@ def test_main_jsonschema_json_pointer_array():
)
with pytest.raises(SystemExit):
main()
@freeze_time('2019-07-26')
def test_main_use_default_kwarg():
with TemporaryDirectory() as output_dir:
output_file: Path = Path(output_dir) / 'output.py'
return_code: Exit = main(
[
'--input',
str(OPEN_API_DATA_PATH / 'nullable.yaml'),
'--output',
str(output_file),
'--input-file-type',
'openapi',
'--use-default-kwarg',
]
)
assert return_code == Exit.OK
assert (
output_file.read_text()
== (EXPECTED_MAIN_PATH / 'main_use_default_kwarg' / 'output.py').read_text()
)
with pytest.raises(SystemExit):
main()